Andrew Morgan's Blog

It's a blog

GSoC Weekly Progress Report #5

July 04, 2017 — Andrew Morgan

Hello again! Quick report here due to only a few days since the last one.

This period has been mostly focused on writing tests for qvm-file-trust. I got a bit hung up on some of the python-specific stuff but am now plugging away at covering all the methods.

The main difficulty was that the majority of the methods deal with reading from and manipulating the content and attributes of files on the file-system. Ideally the tests shouldn't actually create files on the system to be read, but instead provide testing data to the open() calls of the program.

Functionality to substitute certain python methods with our own are provided by the unittest.mock library, and it's a incredibly useful tool once you wrap your head around it.

The big issues I had while writing tests were having our test data being correctly read by the program, and supporting returning different sets of data for different open() requests within the same method.

The first issue comprised of the unittest.mock.mock_open function not supporting iterators. The qvm-file-trust tool uses the following code to read from the global and local untrusted folder lists:

    with open(GLOBAL_FOLDER_LOC) as global_list:
        for line in global_list:
            ...
            untrusted_paths.add(line)

To dynamically inject the returned content with unittest.mock, we can write the following:

with unittest.mock.patch('qvm-file-trust.open', unittest.mock.mock_open(
    read_data='Return me!')):
    untrusted_folder_paths = qvm_file_trust.retrieve_untrusted_folders()
    ...

Any file then read in the retrieve_untrusted_folders() will now return 'Return me!', instead of the actual file contents.

While simple enough in this case, the contents of our global list file as returned by our test data was always empty, and the inner for loop would never run.

I mulled over this issue for a few hours, when Marek pointed out what was actually happening. It turns out that mock_open does not mock the __iter__ function on the object, and thus our attempts to read line-by-line by iterating over the file fail miserably. This is a recognized bug in python.

To alleviate this, it turns out that we can simply manually override the iteration function and tell it to return our desired lines:

mock_object.return_value.__iter__ = lambda self : iter(
    'home/user/Downloads', '')

We are now able to substitute file content dynamically in our test cases. There was still one problem though. Using the above code, we end up replacing all open() calls with our substituted content. But this method makes multiple calls, to multiple files, which for proper testing requires multiple different returned values.

We need to return different content based on the path we are asked to read.

Figuring this out took another long period of time, but eventually I was able to return multiple values at different calls using mock's side_effect attribute.

This approach was slightly limited in that we can only return data based on when open() was called, rather than what path open() was called with, but as our methods are rather static in this sense there was no trouble.

The resulting code looked like the following:

def test_000_retrieve_override(self, list_mock):
    """Create a mock global and local list and check resulting rules.

    Are global rules are properly overridden by '-' prepended local rules?
    """

    handlers = (unittest.mock.mock_open(
            read_data="/home/user/Downloads\n/home/user/QubesIncoming"
            ).return_value,
            unittest.mock.mock_open(
            read_data="/home/user/Downloads\n-/home/user/QubesIncoming"
            ).return_value)
    list_mock.side_effect = handlers

    untrusted_folder_paths = qvm_file_trust.retrieve_untrusted_folders()

self.assertEqual(untrusted_folder_paths, {'/home/user/Downloads'})

Here we subtitute the global lists rules as

/home/user/Downloads
/home/user/QubesIncoming

and the local lists as

/home/user/Downloads
-/home/user/QubesIncoming

The '-' in the second file should override and negate the rule in the first file. We then check that we end up with only the Downloads folder being left as untrusted, and if so mark the test as passed.

While this took quite a while to figure out, injecting return code from multiple files was necessary for nearly all the tests here, so with that out of the way writing the rest of them should be fairly straight-forward. With a good understanding of how unittest.mock() works, even testing chmod and xattrs should be a breeze.

That about wraps it up for this blog post. Happy America day to anyone celebrating that tomorrow. See you all in a week.

As always, the code is available here.

Tags: gsoc-2017, progress-report