Vim security

Just read https://glyph.twistedmatrix.com/2015/11/editor-malware.html and https://jordaneldredge.com/blog/why-i-switched-from-vundle-to-plug/ which both discuss some issues about text editor security.

TL,DR:

1. vim-plug is preferred to vundle
2. developers may be seen as high value targets for system compromise.
3. beware git:// protocol

A Python Christmas present

Nose is my current Python testing framework of choice.

When I'm doing TDD which is more or less all the time*, I'm capturing logs and (shock horror) print statements and I commonly invoke nosetests in a very noisy way with the -sv switches.

A few days ago, I found myself thinking about writing an output colourizer for nose when (gasp) it occurred to me that one might already exists. And, wonder of wonders, I was right.

I would like to introduce you to rednose. It's nearly Christmas, the plugin (appropriately) is called 'rednose' and it represents the gift of visually interesting test results.

Rednose is easy to install, easy to invoke, and it works like a dream. I was especially pleased with it's ability to colourize logs and stack traces.

Kudos to the developer https://github.com/timbertson.

* If I'm not doing TDD I'm researching an unfamiliar API inside an IPython/Jupyter notebook.

Jupyter notebooks

My typical working setup is a headless GNU/Linux VM which I access via ssh. This means that when I working in a newly initialized project environment and I want to pop up a Jupyter notebook I do this:

pip install jupyter
jupyter notebook

And bang! there I have a w3m text based browser, nicely colourized, but not what I was hoping for.

Every month or so I go through this. And I have been trained to find the Jupyter config file or the IPython config file as was and edit it to suit.

So this time the version of Jupyter which pip harvests for me is 4.0.6. And could I find the configuration? No I could not.

It turns out you have to generate the configuration before you can actually edit it:

jupyter notebook --generate-config

Once that is done, then you can

1. change the host from localhost to '0.0.0.0' and
2. change start browser by default to 'False'.

Oh, and look for the configuration in

.jupyter

.

Probability in business

As someone who has put together quite a few business plans over the years, this short (4 minute) video from Prof Sam Savage struck me as quite insightful.

For me it highlights the difference between those of us with a knowledge of the theory of probability and those who have really thought about the way probability matters manifest in daily life. (I'm somewhere in between, and probably (hmm interesting word choice) closer to the theoretical end).

I notice that in the background, Prof Savage has a copy of Bernstein's "Against the Gods" which I am pleased also to own and to have read, although in any discussion on probability, it would be a crime not to mention "Fooled By Randomness" from Nassim Nicholas Taleb.

And now that I'm thinking of statistical solecisms, the common theme between Savage, Bernstein and Taleb, I'm reminded of the joke about the goverment education minister who was often heard to complain of the large number of schools which were still below average.

Enjoy the video.

Using Python unittest outside a testing environment

Test frameworks are not just for testing code during development and deployment.

Test frameworks may also be put to good use within your application.

By way of example, consider the possibility of running diagnostics on a live application. Another possibility is checking the validity of an on-the-fly configuration change.

The Python unittest library, part of the standard distribution can easily be incorporated into your live code.

The code example below presents a toy example of a test case, which is then exercised by your application process. The outcome is a test result object which you can query and report on.

import unittest


class TestMe(unittest.TestCase):
    def test_this(self):
        self.assertTrue(False)
    def test_that(self):
        self.assertTrue(True)

loader = unittest.TestLoader()
suite = loader.loadTestsFromTestCase(TestMe)
result = unittest.TestResult()
suite.run(result) # run actually returns result as well as populating it
print result

<unittest .result.testresult="" errors="0" failures="1" run="2">

The variable 'result' is an instance of unittest.TestResult() and has a rich set of attributes and methods as documented here.

Python3 Namespace Packages Using Wheels

I was experimenting with Python3.4 and namespace packages.

Here's what I did.

I created two completely separate repos, A and B.
I had a top level namespace, toplevel.

I created a directory structure:

repoA/toplevel/libA/moda.py
repoA/setup.py

repoB/toplevel/libB/modb.py
repoB/setup.py

The setup.py files were very similar:

from setuptools import setup

setup(
    name = "toplevel libA",
    version = "0.1",
    packages = ['toplevel.libA']
)

and:

from setuptools import setup

setup(
    name = "toplevel libB",
    version = "0.1",
    packages = ['toplevel.libB']
)

To be able to work with the bdist_wheel extension to setuptools, I had to use pip to install wheel.

pip install wheel

Then in repoA and repoB it was possible to execute:

python setup.py bdist_wheel -d /tmp/wheels

In this way, two wheels were created:

/tmp/wheels/toplevel.libB-0.1-py3-none-any.whl
/tmp/wheels/toplevel.libA-0.1py3-none-any.whl

I then created a third project repo which used these two wheel packages.

repoC/someotherproject

In this repo, after creating and virtual envirionment, it was possible to install the two wheels:

pip install /tmp/wheels/toplevel.libB-0.1-py3-none-any.whl
pip install /tmp/wheels/toplevel.libA-0.1py3-none-any.whl

In the site-packages directory of the repoC environment there was a tree structure installed:

./toplevel/liba
          /libb

Showing that libA and libB, although separately developed, were available in the client project as a unified namespace. (Note that liba and libb could have been independently installed on different branches of sys.path and they would stilll appear to the interpreter as an integrated whole.)

Getting started with Sphinx

I really like pdoc as a great lightweight python source code documentation tool.

I like to invoke it:

PYTHONPATH=. pdoc --http --http-host 0.0.0.0 --http-port 8888 --only-pypath

I like to use ReST to format docstrings. Unfortunately I have not been able to get pdoc to display parameter lists as I would hope.

(UPDATE: 20151205: pdoc will honour parameter lists created using Markdown.)

So I switched to using Sphinx. It seems I am not alone in finding the Sphinx documentation hard to use.

After a bit of futzing around, this recipe meets my objective of having nice documentation with the power of Sphinx but the ease of pdoc.

1. Install Sphinx.
2. Invoke sphinx-apidoc -F -o dox
3. cd to the dox directory.
4. Edit conf.py to ensure that sys.path can find the module to be document.
5. make html
6. python3 -m http.server

I hope it works for you.