set your code free

releasing and maintaining an open-source Python library

Carl Meyer

Hi! Welcome to the final talks session of PyCon
I know it's been a long conference
I'll at least keep this short
If I'm lucky also keep you awake
and send you off on a high note, inspired to release your own software!

You have code!

You want me to use it (pip install).
You want me to contribute to it.

The premise of this talk is that you have some code.
Writing that code is out of scope.
--
pip install
not web app deployment or GUI installer
--
"accept my contributions without breaking your software or losing your sanity"
So you look up the docs on how to do this...

...and you find lots of different projects
all with their own documentation to read
When you're doing something for the first time, choices kill.
This talk presents a set of rails for setting up your first open-source project..
It's not the only way, but _a_ way that will work.
Long on opinions, short on choices.
"From zero to awesome in 20 minutes."
The awesome:
public HTML docs with built-in navigation that update when you push
tests that run on every push (and on pull requests)
pip install ready
welcoming to contributors.
If you've done this before, hopefully you'll still pick up a new trick or two.

All the things

Project structure.
Choosing a license.
Code hosting.
Documentation.
Testing & CI.
Packaging.
Community.

The roadmap for this talk.

01. Structure

 1  .
 2  └── PyFly/
 3      ├── docs/
 4      ├── pyfly/
 5      │   └── __init__.py
 6      ├── tests/
 7      ├── LICENSE.txt
 8      ├── MANIFEST.in
 9      ├── README.rst
10      └── setup.py

This is the bare bones;
We'll flesh this out and add to it as we go.

02. License

The conditions for my use.
No license means I can't use it.
If you aren't sure, use BSD or MIT.
Or (L) GPL, Apache, MPL.
Don't use anything else.
H/t: @jacobian 's lightning talk.

First decision: releasing software as open source means choosing a license.
I am not a lawyer, this is not legal advice.
--
Your license is the conditions under which I can use your code.
--
If you don't have a license, the default is "all rights reserved."
A project without a license is not open-source, even if its on GitHub!
--
BSD or MIT are unrestrictive licenses;
All you ask from your users is that they credit you: keep your name and the license attached to your work.
--
GPL is more restrictive; requires that any work derived from yours must also be released as GPL. If you're worried about freeloaders, you can go this route, at the cost of having fewer users.
Apache and MPL are reasonable choices if you know why you're choosing them.
--

`LICENSE.txt`

 1  Copyright (c) 2009-2014, Carl Meyer and contributors
 2  All rights reserved.
 3 
 4  Redistribution and use in source and binary forms, with or without
 5  modification, are permitted provided that the following conditions are
 6  met:
 7 
 8      * Redistributions of source code must retain the above copyright
 9        notice, this list of conditions and the following disclaimer.
10      * Redistributions in binary form must reproduce the above
11        copyright notice, this list of conditions and the following
12        disclaimer in the documentation and/or other materials provided
13        with the distribution.
14 
15  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
16  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
17  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
18  A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
19  OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
20  SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
21  LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
22  DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
23  THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
24  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
25  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

03. Code hosting

GitHub.
That's where the people are.

We have a lot of areas to cover,
--
so fortunately some are short and sweet.

04. Docs

If it's not documented, it doesn't exist.
Build it with Sphinx.
Host it at ReadTheDocs.

--
I like reading code. I will read code to find a bug, to fix a bug, or to better understand how your library does what it does.
But if I have to read your code to figure out how to use your thing -- I'm gonna choose a different library, or just write it myself instead.
Auto-generated API docs don't count, unless your API is very simple. A long list of functions and classes with their docstrings is something I can get from reading your code.
How to write your docs: fortunately another easy choice.
--
And so is where to host it.
The combination of Sphinx and Read The Docs makes it so ridiculously easy to put beautiful, usable docs online, it's a shame not to take advantage of that by writing some!

 1 $ pip install sphinx
 2 ...
 3 
 4 $ cd docs/
 5 
 6 $ sphinx-quickstart
 7 ...
 8 
 9 Enter the root path for documentation.
10 > Root path for the documentation [.]:
11 
12 ...

You can write some sphinx docs in about as much time as I'll spend on the next two slides.
--
pip install sphinx
--
switch to the docs directory
--
run 'sphinx-quickstart'
--
answer some questions; the defaults will do

`docs/index.rst`

 1 Welcome to PyFly!
 2 =================
 3 Installing
 4 ----------
 5 Install **PyFly** with
 6 ``pip install PyFly``.
 7 
 8 Usage
 9 -----
10 Find a route::
11 
12    import pyfly
13    route = pyfly.Route('KRAP', 'CYUL')

If you haven't written restructuredtext, the basics are very easy.
--
underlined headers
--
different levels of headers
--
inline formatting: strong with double asterisk
--
inline code literals
--
code blocks, automatically syntax highlighted in pretty much any language
If you take five minutes and write exactly this much documentation -- a simple usage example -- you've already made your package much more attractive than one without docs.

`make html`

Run 'make html' to generate an HTML version of your docs

Automatically updates the docs every time you push to the repo.
Can build multiple different versions (by branch or tag) and provides a version switcher to choose between them.
Good-looking, mobile-responsive theme.
Win!

05. Testing

If it's not tested, it's broken.

If code is changing over time, and you don't have automated tests for it (or a lot of time on your hands for testing manually), over time the likelihood of that code being broken approaches 1.
Tests are good for any code, but they are critical for open-source code that is getting contributions. Finding time to handle pull requests is hard enough, you really don't want to have to run through a bunch of manual tests for every pull request to verify that it didn't break things.

Versions	Python
Django	pypy	2.7	3.2	3.3	3.4
1.4.10
1.5.5
1.6.2
1.7-alpha
master

A reasonable support matrix for a popular Django add-on library.

Could be worse: with another dependency or two it would have 3 or 4 dimensions, not just 2.

25 boxes in that matrix. Are you gonna create 25 virtualenvs and run the tests 25 times for every pull request to your project? If not, your claim to support all those versions is purely theoretical, and almost certainly not true.

Thankfully, there's a tool to help with this: ...

tox saves the day

Creates a bunch of virtualenvs.
Runs your tests in each of them.

With one command...
--
...

`tox.ini`

1 [tox]
2 envlist = py27,py34,pypy
3 
4 [testenv]
5 deps = pytest
6 commands = py.test

A very simple tox setup: just three Python versions, no dependencies.
Originally I had just 2.7 and 3.4, but I made the mistake of letting Alex Gaynor see my slides, and he tied me up and wouldn't let me go until I added PyPy.
There's a lot more you can do here, such as adding various versions of dependencies in various envs (to handle the matrix we just saw).
You can look at the documentation, or I'd be happy to show you some examples afterwards.

 1 $ tox
 2 GLOB sdist-make: /.../PyFly/setup.py
 3 py27 create: /.../PyFly/.tox/py27
 4 py27 installdeps: pytest
 5 py27 inst: /.../PyFly/.tox/dist/PyFly-0.1.zip
 6 py27 runtests: commands[0] | py.test
 7 ================== test session starts ====================
 8 platform linux -- Python 2.7.6 -- py-1.4.20 -- pytest-2.5.2
 9 collected 3 items
10 
11 test_routes.py ...
12 
13 ================== 3 passed in 0.02 seconds ===============
14 
15 ... <same for py34 and pypy>...
16 
17 __________________ summary ________________________________
18   py27: commands succeeded
19   py34: commands succeeded
20   pypy: commands succeeded
21   congratulations :)

Running your tests

all the time

You get a pull request, you open a terminal, you add the source of the PR as a remote, you pull their branch, you run tox... wouldn't it be nice if when you first looked at the pull request, it already told you whether the tests passed or not?

This used to be hard. Today it is easy.

travis-ci.org

Will do this for free for public GitHub projects.

(There's also drone.io and probably others; Travis is the one I've used.)

To set it up, just go to travis-ci.org and sign in with your GitHub account.
It'll show you a list of all your public GitHub projects, and you just pick which ones you want to enable.
It will automatically set up a GitHub webhook for the projects you enable.

`.travis.yml`

 1  language: python
 2 
 3  python:
 4    - 2.7
 5    - 3.4
 6    - pypy
 7 
 8  install:
 9    - pip install pytest
10 
11  script:
12    - py.test

The second thing we need to do is add a .travis.yml file to the repo, so Travis knows how to run our tests.
This example does the same thing as our earlier tox.ini.
...

Simple example of a Trav

06. Packaging

Oh yes, you may want people to be able to install your thing!

`setup.py`

 1  from setuptools import setup
 2 
 3  with open('README.rst') as fh:
 4      long_description = fh.read()
 5 
 6  setup(
 7      name='PyFly',
 8      version='0.1.2',
 9      description='Flying with Python',
10      long_description=long_description,
11      author='Carl Meyer',
12      author_email='carl@oddbird.net',
13      url='https://github.com/oddbird/PyFly/',
14      packages=['pyfly'],
15      install_requires=['six'],
16      classifiers=[
17          'Development Status :: 3 - Alpha',
18          'License :: OSI Approved :: BSD License',
19          'Programming Language :: Python',
20          'Programming Language :: Python :: 2.7',
21          'Programming Language :: Python :: 3',
22          'Programming Language :: Python :: 3.3',
23          'Programming Language :: Python :: 3.4',
24      ],
25  )

`MANIFEST.in`

include AUTHORS.rst
include CHANGES.rst
include LICENSE.txt
include MANIFEST.in
include README.rst
recursive-include docs *.rst

When Python creates a source distribution of our package, it will include all of our Python packages, plus any additional files we list here.

git tag v0.1.2
git push --tags

First thing to do is tag your release in git
and make sure you push that tag to GitHub
(git doesn't push tags by default)

python setup.py sdist
pip install dist/PyFly-0.1.2.tar.gz
pip install twine
twine upload dist/PyFly-0.1.2.tar.gz
pip install PyFly
Win!

"sdist" == "source distribution"
This is the most common format for distributing Python code, and it's a good choice for pure Python (no compiled extensions).
There is a new wheel format getting a lot of buzz, and it is very exciting especially for Python projects with compiled extensions, but I'm not going to cover it here; for your first project if it's pure Python code I recommend just starting with sdist.
python setup.py sdist will create an sdist for your package in the dist/ subdirectory.
--
Before we upload this sdist to the package index, we want to be sure it works. We can use pip to install directly from that sdist and test to make sure it works correctly.
--
There is a python setup.py command to upload your sdist to the package index, but it uploads it over a non-SSL connection, so we'll instead use the twine tool to do it securely.
--
Before you can do this step, you need to go to the package index in your web browser to create an account and claim your package name.
Then we run twine upload to create the new version, upload its metadata and the sdist file.
Using --sign will sign your upload with your GPG key. This is a good idea, but if you don't have a GPG key you can leave that out.

For more

packaging.python.org
IRC: #pypa on FreeNode
distutils-sig@python.org mailing list

Python Packaging User Guide
#pypa IRC channel on FreeNode
the distutils-sig mailing list

07. Community

Making life better for people using and contributing to your software.
Valuing your users and contributors' time.

Semantic Versioning

X.Y.Z
increment:
X for breaking changes.
Y for backwards-compatible feature additions.
Z for bug fixes.
semver.org

One way to value your users' time is to communicate clearly about changes that will affect them.
With semantic versioning, you can use your version numbers to communicate this information to them.
--
...

Keep a changelog

Semantic versioning tells your users about the magnitude and type of changes in a release.
A changelog tells them exactly what has changed.

`CHANGES.rst`

CHANGES
=======

master (unreleased)
-------------------

2.0.3 (2014.03.19)
-------------------

* Fix ``get_query_set`` vs ``get_queryset``
  in ``PassThroughManager`` for Django <1.6.
  Thanks whop, Bojan Mihelac, Daniel Shapiro,
  and Matthew Schinckel for the report;
  Matthew for the fix. Merge of GH-121.

* Fix ``FieldTracker`` with deferred model
  attributes. Thanks Michael van Tellingen.
  Merge of GH-115.

A sample of what a changelog can look like.
People think "oh, I'll just autogenerate it from my git commit history!"
NOT the same as a git commit log.
Pulls out and highlights changes that are relevant to users.

Have a CONTRIBUTING document

Save time for both your contributors and yourself by having a CONTRIBUTING document.

`CONTRIBUTING.rst`

How to get set up for development.
How to run the tests.
What to include in a bug report.
Coding standards, test coverage standards...

Keep the

tests passing

I'm all excited to fix a bug I've found in your project.
I download it, follow your contributing guidelines to get set up
All excited to run the tests... and some of them fail.
My motivation to contribute is now gone.

Give

quick feedback

Another way to value your contributors' time is to respond quickly.
You may not have time to deal with the issue right away, but you can at least post a quick note thanking them for the contribution and saying you'll get to it later.

Give credit

When people pitch in, give them props!
In the commit message, in the changelog, in an AUTHORS file...
Motivates people to contribute.
Give commit access to helpful contributors!
Not as technically important with DVCS as it used to be, but a mark of confidence, will motivate greater involvement (and saves you work).
Someday you'll want to hand off maintenance...

Be nice

Anytime you get a bug or pull request, even if it's irritating or someone hasn't done their homework, it represents someone investing time and energy in your project. Thank them for their time and energy!
For every one person you communicate with directly in a public way, there are 10, 20, 50, 100 people watching that communication, now or later. How you treat contributors will affect their motivation to contribute.

Questions?

oddbird.net/set-your-code-free-preso

Space	Forward
Left, Down, Page Down	Next slide
Right, Up, Page Up	Previous slide
P	Open presenter console
H	Toggle this help