Notes on Ecto Migrations

I got tripped up recently when trying to write my first Ecto migration.  The style in Ecto changed from specifying migrations as DDL strings to a database-agnostic style via macros and functions.  The problem I was running into is that most tutorials/posts refer to the older string-based style.   I was using a newer version of Ecto with DDL strings and my tables weren't being created.

The goal of this post is to give a basic overview of the new style and give some examples of its usage.  All examples assume Postgres as the target database.

In the new migration syntax, tables are created with the create macro and columns are specified using the add function.  An example of an up migration:

def up do
  create table(:table_name) do
    add :column1, :string
    add :column2, :integer
    ...
  end
end

The table's name is specified as an atom.  Columns are defined by the add function:

add(column, type \\ :string, opts \\ [])

where the first argument is the name of the column (an atom), the second argument is the type of the column (also an atom), and the remaining arguments are passed as options (more on this below).  Ecto provides the following basic type atoms (the resulting Postgres column type follows as a comment):

:integer  # integer
:float    # double precision
:boolean  # boolean
:string   # character varying
:binary   # bytea
:uuid     # uuid
:decimal  # numeric
:datetime # timestamp without time zone
:time     # time without time zone
:date     # date

Arrays are supported as composite-types having a tuple of :array and type atoms:

{:array, :string}  # character varying[]
{:array, :integer} # integer[]

The default type for the add function is :string, which results in an unbounded character varying type in Postgres.  So this:

add :post_body  

is the same as this:

add :post_body, :string

The add function also accepts arbitrary atoms, allowing for database-specific types.  Instead of using :string, :text could be used for Postgres's text character type:

add :post_body, :text  # text instead of character varying

The add function's opts argument accepts the following options:

:primary_key
:default
:null
:size
:precision
:scale

Option Examples:

By default, Ecto will create a serial primary key column for your table with a name of "id".  You can override this behavior by passing "primary_key: false" in the table function and passing the  "primary_key: true" option in a column's add function:

create table(:table_name, primary_key: false)
  add :doc_id, :uuid, primary_key: true
end

Other options:

Making a column not-null (the default is to allow nulls):

add :title, :string, null: false

Giving a column a default value:

add :author, :string, null: false, default: "Anonymous"

Limiting the size of a column:

add :state_cd, :string, size: 2  # character varying(2)

Setting precision and scale:

add :latitude, :decimal,  precision: 14, scale: 11 # numeric(14, 11)

Other:

Ecto will add inserted_at and updated_at columns if you add a "timestamps" call in the body of the create macro:

create table(:table_name)
  add :title, :string
  
  timestamps
end

Note that in Postgres, these are created as "timestamp without time zone" columns that do not allow nulls and do not have a default value.   A default value can be specified in your model.

There's a lot of stuff missing from this post, but it should be enough to get you up and running with the new migration style.  Ecto is still young and changing.  When in doubt, consult the source.

Recommended F# Beginner Resources

It's still very early in my F# journey, but after spending some time with several resources available to newcomers, I have some recommendations.  I believe the following list to be a decent path for moving from Novice to Advanced Beginner in F#.  If you think I've missed some gems or disagree, please leave a comment.


Try F#

This site takes the "Try [insert language]" idea that has been done in other languages and really runs with it, providing several topic-oriented tutorials beyond the basics.  It's a great way to get your brain prepped for F#.   Note:  I had some issues getting the script window to echo what I was typing in Safari on Mac OSX.  I had to switch to Chrome in order to get it to work.


The F# Survival Guide

Even though it is dated, I consider this the best tutorial on F# for a beginner.  The concepts are introduced in a logical progression that build on top of one another, the pacing feels right, the language used is simple and direct, and code is introduced in a way that can be run directly in fsi (a recurring theme in this post is learning by doing).   After getting discouraged reading a couple of books where the authors failed to trim the context down in their examples or would bounce around concepts in a strange order, this resource gave me confidence.  Once I finish this, I plan on going back to the books I gave up on because they are more current than the survival guide.

For some reason, the F# Survival Guide's original website is no longer available, but the contents can still be found on the Internet Archive.   It's obvious the authors put a lot of effort into this excellent resource.   It's a shame it doesn't have an official home or the ability to be updated.


F# Koans

Many languages now have Koans, but the basic premise is the same:  given a series of tests that cover the language's fundamentals, fill in blanks in the code that makes the tests pass.   Starting from simple basics and moving to more complex concepts, the Koans are a great way to have simple problems put in front of you that require thinking in the language to solve.


Exercism.io

I recommend not attempting these until you have some F# practice under your belt.  Similar to the Koans, Exercism exercises present you with failing test cases that you must write code to fix.  The difference from the Koans is that the exercises are not F#-specific and you can get feedback from mentors when you submit a solution.  I wrote more about Exercism with F# in a post that includes how to get set up using Xamarin Studio on Mac OSX.

This one suffers a bit due to the size of the F# community participating.   None of the F# exercises I've submitted have gotten feedback.  Other languages have more mentors willing/able to provide feedback, and it does make a difference.   Maybe a combination of attempting these and checking in with the F# irc channel would suffice?


Domain Modelling with the F# Type System

This fantastic talk by Scott Wlaschin opened my eyes to algebraic types.  Scott has a knack for taking complex concepts and making them accessible.  Watching this talk made me feel like I've got a lot to learn (more fun ahead).  It brings up some questions:
  • What would business logic/service layers look like using these types? 
  • How would these relationships be stored in an RDBMS?
  • What are the best practices for structuring large F# programs?

Which leads me to the following...

Continuing Education


Once you've read/watched the above, you should be comfortable with a lot of the F# that you will encounter.  It's at this point where you enter a different phase:  knowing enough to be dangerous but feeling ill-equipped to tackle more serious problems.

For myself, I'm planning on building a couple of pet projects and writing about them, but I also need more directed learning.   Something along the lines of A Recipe for a Functional App on F# For Fun and Profit (also by Scott Wlaschin, mentioned above).  If you are reading this and have  any recommendations, I'd appreciate them.   Any books, talks?   Are there any non-trivial F# code bases out there that exhibit best practices and are ideal for learners?   Please share!

If you are working on something targeted at teaching F# newbies and need a guinea pig or some feedback, let me know--I love learning this stuff and am interested in contributing.

Using Exercism.io to Grok F#

Since I'm at the beginning of learning F#, I'm looking for different ways to get up to speed as quickly as possible and test what I've learned.  To see how well I'm progressing, I've started doing the F# exercises on Exercism.io.

Exercism, created by Katrina Owen, provides learners a way to test their language comprehension by solving programming problems and getting feedback/mentoring by others.  It's like a remote code review.  After using it for a couple of languages and getting some insightful feedback, I've become a big fan.

Here's how it works:  sign up for an Exercism account, download the Exercism command line tool, fetch an exercise in the target language,  solve the exercise, then submit it to the Exercism server.  Once your solution has been published to the server, it is viewable by others who can mark it as "Looks Great", or give a "Nitpick".   You are notified of any feedback.   If the feedback suggests improvements, you can make a new revision and re-submit, or defend your choices and mark your solution as "Done."  Once you have submitted a solution for a given exercise, you can also browse other member's solutions to that same exercise and leave feedback of your own.  It's interesting to see how differently  people solve the same problem, and to contrast other solutions with your own.

An Exercism exercise consists of at least two files:  (1) a Markdown-formatted README and (2) a test file.  The README explains the exercise and the test file contains several tests written in the target language. You create a source file in the target language that makes the tests in the test file pass.   Once the tests pass, you submit the solution, then fetch the next exercise and repeat.

If you're at that point where you know enough F# to be dangerous but aren't sure where to go next, I highly recommend attempting the exercises on Exercism.io.  The more people use it, the more value it will bring those seeking to learn.  Feel free to log in and give me some Nitpicks--I can take it!

The remainder of this post walks through how I set my environment up (Mac OSX + Xamarin Studio) to do the Exercism problems.  I added some tips and the end of the post that cover some problems I experienced.


Download Xamarin Studio


I wrote about this in a previous post.


Sign up/install Exercism


Sign up for an Exercism account (requires a GitHub login) and follow the instructions to get started and download the command line tool.  Make sure you have already run exercism fetch fsharp (to download the first exercise for just F#), or exercism fetch (to download the first exercise for all languages).  The remaining steps assume you have the exercism command-line tool set up and have the exercism directory containing at least the first exercise ready to go.


Create a new F# Library Project in Xamarin Studio


Run Xamarin Studio, then click the "New Solution..." button, choose "F# Library", and give your project a name:





Install the NUnit and NUnit Runners packages


Select the "Project" menu, then the "Add Packages..." menu item.  When the "Add Package" window appears, type "NUnit" in the search field to limit the package results (there are a lot), then check the "NUnit" and "NUnit.Runners" checkboxes and click the "Add Package" button:





Add the first exercise's test file


When you ran the exercism fetch command, it should have created an exercism directory with an fsharp subdirectory.  This is directory Exercism will fetch exercises to.  Each exercise gets its own subdirectory.  Inside the fsharp subdirectory, you should see a sum-of-multiples subdirectory.  This is the first exercise.

Inside the sum-of-multiples subdirectory, you should see 2 files:
  1. README.md
  2. SumOfMultiplesTest.fs
The README file is a Markdown-formatted text file that explains the exercise and gives some basic tips on how to solve it.  The SumOfMultiplesTest.fs file is an F# source file that contains several NUnit tests that your solution file must pass before submitting.   The test file should be added to the  project.

Right-click on the project's name (not the solution's name), then choose "Add", then "Add Files...".  In the "Add Files" window, navigate to the sum-of-multiples directory and select the SumOfMultiplesTest.fs file:





Solve the exercise


If you open the SumOfMultiplesTest.fs file in Xamarin Studio, you should see some errors:




Your goal for the exercise is to: (1) make the errors go away, and (2) make all the tests in SumOfMultiplesTest.fs pass.  To do this, you'll need to create an F# source file that contains the missing functions and that fulfills the obligations of the tests.   You can run the tests by selecting the "Run" menu, then the "Run Unit Tests" menu item.

Once all of the tests pass, copy your solution file back into the exercism/fsharp/sum-of-multiples directory, cd into that directory and type exercism submit [solution filename].  This will submit your solution to the Exercism.io server and allow it to be viewable by others.  If you run the exercism fetch command again, it will pull down the next exercise.  


Tips


The F# exercises require the answers to be written in the form of a module and a class containing the solution functions.  Before attempting the problems, you should understand how F# classes work.  This post on the awesome F# for Fun and Profit should help.

The ordering of files in an F# project matters.  Your solution file should come before the test file.  You can re-order files by clicking on one, then dragging it above/below other files.

The test files have the [<Ignore>] attribute set on all but the first test.   Once you solve a  test, remove the Ignore attribute from the next test and run the tests again or it will show up ignored in the Test Results window.


Good Luck! 



If you see anything wrong with the post or have comments, please let me know here or on twitter.