Engineering

Learning to Sight Along the Space Capsule

Testing does not have to be complicated, but sometimes it can turn into spaghetti just like any other code. This talk by Sandi Metz at Railsconf 2013 inspired me to revisit some of the unit tests for one of our services that relies heavily on FTP and SFTP connections.

Why are my tests failing?

The previous test suite for this service used a gem that was originally created as a mocked framework for testing FTP connection behaviors. However, a few key problems existed with this solution:

  • The gem hasn’t been maintained in a couple years, which started to cause intermittent spec failures.
  • The gem doesn’t support SFTP connections, which our service relies much more heavily on than FTP.
  • The gem doesn’t actually mock anything. It starts up a local FTP server, runs the tests, and then shuts the server down. Each test that’s run against it is generating a real response, albeit locally. This not only took up a lot of time (starting and stopping the local server before and after each spec), but was also another source of intermittent failures.

So, after creating a plan of attack, I rewrote or deleted every single test that used this gem. Each test was also re-evaluated for “sighting along the space capsule,” as Sandi Metz calls it. Any test that was written for a private method, a message sent to the object’s self, or an implementation detail was scrutinized (and most of them deleted), which exposed the true underlying issue: almost every single test we’d written that used the gem was an implementation test, not an interface test.

External responses are supposed to be completely mocked out in unit testing, not make actual calls to an API or server. Tests exist to make sure our code works as expected, not to verify that no errors are raised when making external requests. A unit test that requires a response from an outside source is already a broken test; save those for integration tests instead.

Reframing our point of view

Here’s an example to illustrate my point. Let’s say we have an object whose job is to get a list of files from a specific folder on an FTP server. For brevity’s sake, let’s define it like this, with the nitty gritty details fleshed out elsewhere:

class FtpThing
  def get_list_of_files(username, password, folder_name)
    @some_connection.open(username, password)
    list_of_files = @some_connection.list(folder_name)
    @some_connection.close
    list_of_files
  end
end

Now, one option for testing this is the way we did it before, something like this:

let(:ftp_thing) { FtpThing.new }
let(:list) { [‘file1.txt’, ‘file2.txt'] }

before do
  # spin up the “mock" (real) server
  # put a (real) file or two in a folder on the server
end

it ‘gives me the right list of files’ do
  expect(ftp_thing.get_list_of_files(‘username’, ‘password’, ‘folder_name’)).to eq(list)
  ftp_thing.get_list_of_files(‘username’, ‘password’, ‘folder_name’)
end

after do
  # shut down the server
end

On the surface, this doesn’t look terrible. It’s testing the value returned from getListOfFiles, right? Yay, that’s what we want! But… not quite. We’ve started a real server, and we hope that a) it started in time for our tests to run, and b) it doesn’t fail or raise any errors, which is how our tests will pass. Instead, we need to allow a mocked response and set up our expectation this way:

let(:mock_connection) { double(‘ftp’) }
let(:ftp_thing) { FtpThing.new }
let(:list) { [‘file1.txt’, ‘file2.txt'] }

before do
  ftp_thing.set_instance_variable(:@some_connection, mock_connection)
  allow(mock_connection).to receive(:open)
  allow(mock_connection).to receive(:list).and_return(list)
  allow(mock_connection).to receive(:close)
end

it ‘gives me the right list of files’ do
  expect(ftp_thing.get_list_of_files(‘username’, ‘password’, ‘folder_name’)).to eq(list)
  ftp_thing.get_list_of_files(‘username’, ‘password’, ‘folder_name’)
end

See the difference? We aren’t starting a server or making a real external call; we’re just pretending that it happened and gave us the right response. The true test is whether our code is giving us the right value back after it makes an external call. This is also an easy way to test error responses, too! I can allow my connection to raise an error, and then test my error handling from there, rather than trying to fake out a whole server thing.

Bonus points: If I want to change the implementation of my server call—maybe I want to change how it connects or opens the connection, or something similar—then my tests are not broken and I can continue on my merry way.

The moral of the story

No project is complete without some beautiful numerical data to back it up, so here’s some numbers to show off just how much time this saves us. Running the entire test suite on the application before:

Finished in 2 minutes 56.1 seconds (files took 2.72 seconds to load)
172 examples, 0 failures

And after:

Finished in 5.67 seconds (files took 2.77 seconds to load)
155 examples, 0 failures

That’s a 97% time improvement. Precious minutes of time regained, and no more intermittent failures as well.

The moral of the story? Don’t make real calls to real servers in your tests, even if they’re local to your machine or CI container. Set up mocked responses when needed, and test that your code is actually working, not theirs. (And seriously: watch that talk by Sandi Metz at least three more times until it really sticks. Your future self will thank you.)

Jq - the Cool Tool for School

There’s something incredibly satisfying about finding the perfect tool for a job. If you deal with JSON and you haven’t played with jq, you’re in for a real treat.

The description from their site states, 

“jq is like sed for JSON data – you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text… …jq can mangle the data format that you have into the one that you want with very little effort, and the program to do so is often shorter and simpler than you’d expect.”

I’ll go over a few use cases I’ve had for it so far.

Simple Case

The most basic and neat thing you can do with jq is to pipe JSON into jq to pretty-print it using '.'  Whether the JSON is from a curled response:

curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq '.'

or just some JSON you need to use:

cat template.json | jq ‘.’

It’s pretty and readable! This becomes more useful when you’re dealing with messier JSON.

Validation

I’ve also used it as a way to find errors in what I assumed was valid JSON format. My use case has been for when DynamoDB expects imports in a special JSON format. I found out that the import failed, but not much more than that. Knowing that the most likely culprit is probably illegitimate JSON somewhere in the mountains of data, we piped it all through jq to find an exact line of where an extra quotation had creeped into one file. Talk about a needle in the haystack.

This is also especially useful when you want to validate JSON and the data could be sensitive information.

Sorting Keys

Have you ever had to diff a large piece of JSON with another JSON only to find out that the keys are in a different order? Your tooling tells you that all of the things are not like the others which ends up not being useful at all. 

My generated CloudFormation template keys were in a completely different order than GetTemplate from the current stack, and I needed a way tell what the delta between the two was. The -S or --sort-keys option will sort the keys of your hash all the way down so that two sorted equivalent hashes will be identical. Knowing that, I was able to create two sorted files and diff them from there.

cat actual.json | jq -S . > asorted.json

cat proposed.json | jq -S. > bsorted.json

Use your favorite diffing tool and the problem resolved in three lines! meld asorted.json bsorted.json

More Info

There are other neat features to jq, such as very powerful filtering. You can find out more in their manual here: (https://stedolan.github.io/jq/manual/)

Rails Active Records: Assigning a Non-boolean to a Boolean

We’ll discuss how Active Records handle data assignment to a numeric attribute. I observed this ‘odd’ behavior while working in my last epic.

We’ll work with boolean attribute here because Mysql treats boolean data as numeric. Mysql doesn’t have inherent data type called ‘boolean’. When create a column of type boolean internally stores the binary state in a ‘tinyint’ (1 byte datatype which holds integer values in the range -128 to 127). TRUE , FALSE are simple constants which evaluate to 1 & 0.

Now let’s imagine Active Record trying to work with a boolean column in mysql. What happens when we assign string data to a boolean attribute in AR. Active Record will try and coerce the data set in the attribute to a number (because boolean is numeric in mysql). Great!!! How does it convert string to int ?

I know 2 different ways this can be achieved in ruby.

1
2
3
4
pry(main)> Integer('23')
=> 23
[4] pry(main)> '23'.to_i
=> 23

Interesting to see the behavior of the above methods when try to cast a non integer to integer.

1
2
3
4
5
pry(main)> 'asdf'.to_i
=> 0
[2] pry(main)> Integer('asdf')
ArgumentError: invalid value for Integer(): "asdf"
from (pry):2:in `Integer'

The #Integer method complains whereas the use of #to_i results in 0. Unfortunately Active Record uses #to_i to set a boolean attribute and results in FALSE for any non boolean assignment. :–(

Here’s what happened :–

1
2
3
4
5
6
7
8
9
10
11
pry(main)> ds = DataSource.find(5)
  DataSource Load (0.3ms)  SELECT `data_sources`.* FROM `data_sources` WHERE `data_sources`.`id` = 5 LIMIT 1
=> #<DataSource id: 5, auto_approval_enabled: true>
[2] pry(main)> ds.auto_approval_enabled
=> true
[3] pry(main)> ds.auto_approval_enabled = 'asdf'
=> "asdf"
[4] pry(main)> ds.save!
=> true
[5] pry(main)> ds.reload.auto_approval_enabled
=> false

The world is bad. Not really ..

We only observe this behavior in Active Record 3. With Active Record 4 it throws the much needed warning for non-boolean to boolean assignment.

1
2
3
4
DEPRECATION WARNING: You attempted to assign a value which is not explicitly `true` or `false` 
("asdf") to a boolean column. Currently this value casts to `false`. This will change to match Ruby's 
semantics, and will cast to `true` in Rails 5. If you would like to maintain the current behavior, you 
should explicitly handle the values you would like cast to `false`.

Much better, right ??

Starting to Work With Databases on Linux

Starting to Work with Database on Linux

It irks and surprises me when I see engineers invoke mysql or postgres on their development box (i.e., localhost) in some funky way. These include: mysql -uroot and psql -Uroot. You should be able to work locally without pretending to be someone else. It’s just weird.

So, do yourself a favor and start right with the correct set of permissions.

MySQL

After you sudo apt-get install -y mysql-server-5.6, and install without a password issue a: mysql -uroot -e"grant ALL on *.* to $USER" mysql

PostgreSQL

After you sudo apt-get install -y postgresql, issue a: sudo su -c "psql -c\"create user $USER with SUPERUSER\"" postgres

Improve Names in Your Code

Clean Code

Clean Code by Robert C. Martin is a classic book that can be helpful to programmers of all experience levels, programming in any language. Since the advice is universal to coding, whenever I read a chapter, I find practices to use the very next time I code. Here are some thoughts from Chapter 2: Meaningful Names. I have highlighted the sub-headings used in the book in bold.

Use Intention Revealing Names – As the author says, “The name of a variable, function, or class, should answer all the big questions. It should tell you why it exists, what it does, and how it is used.” – The names you create in the process of coding can help organize your thoughts. For example, when coding a game of Tic-tac-toe, ‘determine_winner’ can be a placeholder for a method you will need in the future.

Avoid Disinformation – Only use words that have meaning to programmers if you take care to use them correctly. Examples include ‘list’ and ‘factory’.

Make Meaningful Distinctions – Avoid redundant naming, such as ‘username’ in the User class. – If you add to a concept, take care to distinguish the original name from what you have added. For example, ‘address’ and ‘address2’ have more meaning when named ‘mailing_address’ and ‘work_address’.

Use Searchable and Pronounceable Names – As the author says, “If you can’t pronounce it, you can’t discuss it without sounding like an idiot.”

Pick One Word Per Concept – Minimizing these words lends consistency to your codebase. For example, synonyms like ‘fetch’, ‘retrieve’, and ‘get’ should not be littered throughout your classes. Read through a similar file in the codebase to observe the convention, then follow it.

Now What?

These are the concepts that resonate with me. So, how can you use this information when coding tomorrow (or right now)?

  • Ask for help. If you are stumped on a name for a new class or library, talking with a co-worker can help. A quick search online can also help. Recently, I came across StackExchange’s “English Language & Usage” site when searching for a word that describes both ‘encoding’ and ‘decoding’.

  • Comment on pull requests. If you are wondering if a class, method, or test can have a better name, it probably can! You do not need to provide the ultimate solution; raising the concern can lead someone else to brainstorm something better.

  • Improve names as you code. If you add to, or change, a method significantly, ask yourself if the name continues to be suitable. If you encounter a confusing name as you code, take the time to search the codebase and replace with a better name.

Better naming leads to self-documenting code. It clearly communicates your intent and decreases the cognitive load for everyone who works in the codebase. Good luck!