Commit bc5f1484 authored by Jamesie Pic's avatar Jamesie Pic

Restructure the repository

- add tox for easier reproduction of travis builds
- add idempotence to install_client_deps.sh if CI=1
- mov modules into src/
- update setup.py to install everything as separate modules
- add docs and qa tox jobs
- update documentation

Todo:

- merge django-representatives
- merge django-representatives-votes
parent eeb5b064
[run]
omit =
representatives_positions/contrib/*
representatives_recommendations/contrib/*
src/representatives_positions/contrib/*
src/representatives_recommendations/contrib/*
......@@ -3,11 +3,11 @@ celerybeat-*
core/static/libs/*
# libs
static/libs
src/memopol/static/libs/
# settings
memopol/local_settings.py
memopol/config.json
src/memopol/local_settings.py
src/memopol/config.json
# Byte-compiled / optimized / DLL files
__pycache__/
......@@ -18,6 +18,7 @@ __pycache__/
# Virtualenv
memopol_env
.tox
# Installer logs
pip-log.txt
......
......@@ -3,22 +3,24 @@ env:
global:
- DJANGO_DEBUG=True
- DJANGO_SETTINGS_MODULE=memopol.settings
matrix:
- TOXENV=qa
- TOXENV=docs
- TOXENV=py27
language: python
python:
- '2.7'
- '3.4'
services:
- postgresql
install:
- pip install -U setuptools
- pip install -e .[testing]
- travis_retry pip install -U pip
- travis_retry pip install tox
- travis_retry pip freeze
before_script:
- bin/install_client_deps.sh
- psql -c "CREATE USER memopol WITH CREATEDB PASSWORD 'memopol';" -U postgres
- psql -c "CREATE DATABASE memopol WITH OWNER memopol;" -U postgres
script:
- flake8 . --exclude '*/migrations,docs,static' --ignore E128
- py.test memopol memopol_scores
- django-admin migrate
- travis_retry tox
after_success:
- codecov
deploy:
......
......@@ -36,17 +36,17 @@ export DJANGO_DEBUG=True
export DJANGO_SETTINGS_MODULE=memopol.settings
# Run django migration to create database
./manage.py migrate
memopol migrate
# Import sample data
./manage.py loaddata memopol/fixtures/data_sample.json
memopol loaddata data_sample.json
echo
echo "You're all set!"
echo "To start the application run the following from the repository root ($REPOROOT):"
echo " source memopol_env/bin/activate"
echo " export DJANGO_DEBUG=True DJANGO_SETTINGS_MODULE=memopol.settings"
echo " ./manage.py runserver"
echo " memopol runserver"
echo
echo "If you make changes, don't forget to run tests using:"
echo " flake8 . --exclude '*/migrations,docs,static' --ignore E128"
......
......@@ -10,7 +10,7 @@ Authentication in the admin backend
If you haven't already, create a super-administrator
account with command::
./manage.py createsuperuser
memopol createsuperuser
Then use this account to authenticate in the
administration backend located in ``/admin``.
......
......@@ -23,14 +23,14 @@ Customize settings
Create a copy of the example local settings file::
$ cp memopol/local_settings.py.example memopol/local_settings.py
$ cp src/memopol/local_settings.py.example src/memopol/local_settings.py
Edit ``memopol/local_settings.py`` to set directories, database settings and
Edit ``src/memopol/local_settings.py`` to set directories, database settings and
allowed hosts. Setup your WSGI server to serve:
* Static files from the directory specified in the ``PUBLIC_DIR`` setting to the
``/static`` URL
* The memopol/wsgi.py WSGI application
* The ``src/memopol/wsgi.py`` WSGI application
Initial memopol setup
=====================
......@@ -41,7 +41,7 @@ From the repository root, install python dependencies::
Install client libraries::
$ bin/install_client_deps.sh
$ src/memopol/bin/install_client_deps.sh
Setup the database schema::
......
......@@ -38,8 +38,7 @@ To deploy the website, use a command like::
cron-1.4 \
postgresql-9.2 \
-a yourappname \
-e OPENSHIFT_PYTHON_WSGI_APPLICATION=memopol/wsgi.py \
--from-code https://github.com/political-memory/political_memory.git \
-e OPENSHIFT_PYTHON_WSGI_APPLICATION=src/memopol/wsgi.py \
--no-git
This should create an app on openshift. Other commands would deploy it at once
......@@ -87,7 +86,7 @@ Data provisionning
To fill up the representatives database table, either wait for the cron script
to be executed, either do it manually::
$ rhc ssh -a yourappname 'cd app-root/repo/ && bin/update_all'
$ rhc ssh -a yourappname 'cd app-root/repo && bin/update_all'
OpenShift is fun, login with ssh and look around if you're curious, you'll be
able to recreate your app without much effort if you break it anyway.
......
......@@ -44,6 +44,11 @@ Create a python virtual environment and activate it::
$ source memopol_env/bin/activate
Alternatively, use the tox command::
$ tox -e py27
$ source .tox/py27/bin/activate
Clone the repository
====================
......@@ -132,7 +137,7 @@ Database migrations ensure the database schema is up to date with the project.
If you're not sure, you can run them anyway, they won't do any harm. Use the
following command::
$ ./manage.py migrate
$ memopol migrate
Operations to perform:
Synchronize unmigrated apps: django_filters, staticfiles, datetimewidget, autocomplete_light, messages, adminplus, compressor, humanize, django_extensions, constance, bootstrap3
Apply all migrations: legislature, votes, database, admin, positions, sessions, representatives, auth, contenttypes, representatives_votes, taggit
......@@ -153,7 +158,7 @@ Provision with data
You can load a small data sample for quick setup:
$ ./manage.py loaddata memopol/fixtures/small_sample.json
$ memopol loaddata small_sample.json
Or actual data (takes a while)::
......@@ -164,7 +169,7 @@ Run the development server
Run the development server::
$ ./manage.py runserver
$ memopol runserver
Performing system checks...
......
......@@ -10,12 +10,28 @@ quickstart
<https://www.packtpub.com/application-development/continuous-delivery-and-devops-%E2%80%93-quickstart-guide-second-edition)>`_,
and I bet you'll order a paperback edition for reference !
Testing
=======
Use the ``tox -l`` command to list tests::
$ pip install tox
$ cd political_memory/
$ tox -l
Use the ``tox -e`` command to execute a particular test suite::
$ tox -e py27
And use the ``tox`` command without argument to execute all test suites,
exactly like in CI.
Adding random recommendations
=============================
::
$ ./manage.py shell
$ memopol shell
In [1]: from representatives_votes.models import Proposal
In [2]: from votes.models import Recommendation
In [3]: import random
......@@ -32,14 +48,14 @@ test run is.
To create test fixtures for representatives_positions, insert some Position
objects, and reduce the database with::
./manage.py remove_representatives_without_position
./manage.py remove_groups_without_mandate
./manage.py remove_countries_without_group
memopol remove_representatives_without_position
memopol remove_groups_without_mandate
memopol remove_countries_without_group
For representatives_recommendations::
./manage.py remove_proposals_without_recommendation
./manage.py remove_dossiers_without_proposal
./manage.py remove_representatives_without_vote
./manage.py remove_groups_without_mandate
./manage.py remove_countries_without_group
memopol remove_proposals_without_recommendation
memopol remove_dossiers_without_proposal
memopol remove_representatives_without_vote
memopol remove_groups_without_mandate
memopol remove_countries_without_group
[pytest]
DJANGO_SETTINGS_MODULE=memopol.settings
addopts = --cov-config .coveragerc --cov=. --create-db
addopts = --cov-config .coveragerc --cov=src --create-db
from setuptools import setup
from setuptools import setup, find_packages
setup(name='political-memory',
version='0.0.1',
description='OpenShift App',
packages=['political_memory'],
package_dir={'political_memory': '.'},
description='Political Memory Project Memopol',
packages=find_packages('src'),
package_dir={'': 'src'},
include_package_data=True,
zip_safe=False,
author='James Pic, Laurent Peuch, Arnaud Fabre, Nicolas Joyard',
author_email='cortex@worlddomination.be',
url='http://github.com/political-memory/political_memory/',
......@@ -49,6 +51,7 @@ setup(name='political-memory',
'console_scripts': [
'memopol_import_positions = representatives_positions.contrib.import_positions:main', # noqa
'memopol_import_recommendations = representatives_recommendations.contrib.import_recommendations:main', # noqa
'memopol = memopol.manage:main',
]
}
)
......@@ -24,6 +24,10 @@ function downloadFromGithub()
local repo=$2
local ref=$3
if [ -d $dest ]; then
echo "* $repo ($ref) already downloaded, skipping"
return
fi
echo "* Downloading $repo ($ref) from Github..."
wget -O temp.zip -q https://github.com/${repo}/archive/${ref}.zip
unzip -q temp.zip
......@@ -33,7 +37,9 @@ function downloadFromGithub()
set -e
[ -d ${DEST} ] && rm -r ${DEST}
if [ -z ${CI-} ]; then
[ -d ${DEST} ] && rm -r ${DEST}
fi
mkdir -p ${DEST}
downloadFromGithub jquery jquery/jquery 2.1.4
......
......@@ -2,9 +2,11 @@
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "memopol.settings")
def main():
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "memopol.settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
if __name__ == "__main__":
main()
......@@ -20,7 +20,7 @@ from django.utils.crypto import get_random_string
# Base memopol settings
#
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
DEBUG = os.environ.get('DJANGO_DEBUG', 'false').lower() == 'true'
#
......