Commit d6108022 authored by Jamesie Pic's avatar Jamesie Pic
Browse files

Project modernization

- continuous integration with travis,
- continuous deployment with openshift,
- add update_score cron,
- sentry for exception tracking,
- continuous documentation build with rtfd,
- configuration made 12factor-ish,
- make use of new django-representatives version for database
  optimization.
parent 40829d7d
*.sqlite3
celerybeat-*
core/static/libs/*
deploy
# libs
static/libs
......@@ -21,3 +20,6 @@ __pycache__/
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
.dpl
db.sqlite
log/
The OpenShift `python` cartridge documentation can be found at:
http://openshift.github.io/documentation/oo_cartridge_guide.html#python
For information about .openshift directory, consult the documentation:
http://openshift.github.io/documentation/oo_user_guide.html#the-openshift-directory
For information about action hooks, consult the documentation:
http://openshift.github.io/documentation/oo_user_guide.html#action-hooks
export OPENSHIFT_PYTHON_WSGI_APPLICATION=memopol/wsgi.py
#!/bin/bash
# This deploy hook gets executed after dependencies are resolved and the
# build hook has been run but before the application has been started back
# up again. This script gets executed directly, so it could be python, php,
# ruby, etc.
set -xe
source ${OPENSHIFT_HOMEDIR}app-root/runtime/dependencies/python/virtenv/bin/activate
cat ${OPENSHIFT_REPO_DIR}requirements.txt
pip install -U pip
pip install -r ${OPENSHIFT_REPO_DIR}requirements.txt
# We don't have sentry yet
# python ${OPENSHIFT_REPO_DIR}manage.py raven test
python ${OPENSHIFT_REPO_DIR}manage.py migrate --noinput
pushd ${OPENSHIFT_DATA_DIR}
if ! [ -d node ]; then
wget https://nodejs.org/dist/v4.2.2/node-v4.2.2-linux-x64.tar.gz
tar xvzf node-v4.2.2-linux-x64.tar.gz
ln -sfn node-v4.2.2-linux-x64 node
fi
popd
pushd ${OPENSHIFT_REPO_DIR}
if [ -f ${OPENSHIFT_DATA_DIR}sentry ]; then
pip install raven
./manage.py raven test
else
echo ${OPENSHIFT_DATA_DIR}sentry does not exist, not setting up raven.
fi
PATH="${OPENSHIFT_DATA_DIR}node/bin:$PATH"
HOME=$OPENSHIFT_DATA_DIR
CI=true
npm install bower
npm install
node_modules/.bin/bower install
node_modules/gulp/bin/gulp.js less
mkdir -p wsgi
./manage.py collectstatic --noinput
popd
mkdir -p ${OPENSHIFT_DATA_DIR}media
mkdir -p ${OPENSHIFT_REPO_DIR}wsgi/static/media
ln -sf ${OPENSHIFT_DATA_DIR}media ${OPENSHIFT_REPO_DIR}wsgi/static/media
Run scripts or jobs on a periodic basis
=======================================
Any scripts or jobs added to the minutely, hourly, daily, weekly or monthly
directories will be run on a scheduled basis (frequency is as indicated by the
name of the directory) using run-parts.
run-parts ignores any files that are hidden or dotfiles (.*) or backup
files (*~ or *,) or named *.{rpmsave,rpmorig,rpmnew,swp,cfsaved}
The presence of two specially named files jobs.deny and jobs.allow controls
how run-parts executes your scripts/jobs.
jobs.deny ===> Prevents specific scripts or jobs from being executed.
jobs.allow ===> Only execute the named scripts or jobs (all other/non-named
scripts that exist in this directory are ignored).
The principles of jobs.deny and jobs.allow are the same as those of cron.deny
and cron.allow and are described in detail at:
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/ch-Automating_System_Tasks.html#s2-autotasks-cron-access
See: man crontab or above link for more details and see the the weekly/
directory for an example.
PLEASE NOTE: The Cron cartridge must be installed in order to run the configured jobs.
For more information about cron, consult the documentation:
http://openshift.github.io/documentation/oo_cartridge_guide.html#cron
http://openshift.github.io/documentation/oo_user_guide.html#cron
#!/bin/bash
set -x
cmd=$1
cd $OPENSHIFT_REPO_DIR
export CLEAN=1
nohup bin/update_all > $OPENSHIFT_LOG_DIR/update_all.log 2>&1 &
Run scripts or jobs on a weekly basis
=====================================
Any scripts or jobs added to this directory will be run on a scheduled basis
(weekly) using run-parts.
run-parts ignores any files that are hidden or dotfiles (.*) or backup
files (*~ or *,) or named *.{rpmsave,rpmorig,rpmnew,swp,cfsaved} and handles
the files named jobs.deny and jobs.allow specially.
In this specific example, the chronograph script is the only script or job file
executed on a weekly basis (due to white-listing it in jobs.allow). And the
README and chrono.dat file are ignored either as a result of being black-listed
in jobs.deny or because they are NOT white-listed in the jobs.allow file.
For more details, please see ../README.cron file.
Time And Relative D...n In Execution (Open)Shift!
#!/bin/bash
echo "`date`: `cat $(dirname \"$0\")/chrono.dat`"
#
# Script or job files listed in here (one entry per line) will be
# executed on a weekly-basis.
#
# Example: The chronograph script will be executed weekly but the README
# and chrono.dat files in this directory will be ignored.
#
# The README file is actually ignored due to the entry in the
# jobs.deny which is checked before jobs.allow (this file).
#
chronograph
#
# Any script or job files listed in here (one entry per line) will NOT be
# executed (read as ignored by run-parts).
#
README
For information about markers, consult the documentation:
http://openshift.github.io/documentation/oo_user_guide.html#markers
sudo: false
env:
matrix:
- DEBUG=True
language: python
python:
- "2.7"
- '2.7'
install:
- pip install django
- pip install -U pip
- pip install -r requirements.txt
- cp memopol/config.json.sample memopol/config.json
- pip install django
- pip install -r requirements.txt
before_script:
- npm install -g bower
- bower install
- npm install
- npm install -g bower
- bower install
- npm install
script:
- ./manage.py migrate
- node_modules/gulp/bin/gulp.js less
- ./manage.py migrate
- node_modules/gulp/bin/gulp.js less
deploy:
- provider: openshift
user: jamespic@gmail.com
password:
secure: W7hQDKAtmpOfwLjBuss6NEKqPSrRhsbgH8a8eV+/Oo6HZxMi1mbNFSi+6WRNSs3Cil0ZZV+awoqC61jIzV4oTwEYcy5bv9NWNSY1QO34DECMS5sY00wA0zKhkdsdTr9Pc3TLRp1cw6x2KNCF356FKZojFTRbjtfJ79rqBc5k5ww=
app: dev
domain: memopol
skip_cleanup: true
deployment_branch: pr
on:
repo: political-memory/political_memory
branch: pr
if [ -n "$OPENSHIFT_HOMEDIR" ]; then
source ${OPENSHIFT_HOMEDIR}app-root/runtime/dependencies/python/virtenv/bin/activate
fi
function pipe_download_to_command() {
if [ -n "$OPENSHIFT_DATA_DIR" ]; then
cd $OPENSHIFT_DATA_DIR
fi
[ -n "$CLEAN" ] && rm -rf $1
[ -f "$1" ] || wget http://parltrack.euwiki.org/dumps/$1 || exit 1
if [ -n "$OPENSHIFT_REPO_DIR" ]; then
cd $OPENSHIFT_REPO_DIR
fi
export DJANGO_SETTINGS_MODULE=memopol.settings
unxz -c ${OPENSHIFT_DATA_DIR}$1 | $2
[ -n "$CLEAN" ] && rm -rf $1
}
#!/bin/bash
bin/update_representatives
# grace time for pg
sleep 120
bin/update_dossiers
sleep 120
bin/update_votes
sleep 120
bin/update_scores
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment