Skip to content
Extraits de code Groupes Projets

Comparer les révisions

Les modifications sont affichées comme si la révision source était fusionnée avec la révision cible. En savoir plus sur la comparaison des révisions.

Source

Sélectionner le projet cible
No results found

Cible

Sélectionner le projet cible
  • la-quadrature-du-net/memopol/memopol
  • lnclt/political_memory
  • arthur/political_memory
  • agrausem/political_memory
  • periode/memopol
  • Anthony/memopol
  • Porkepix/memopol
  • jaster/memopol
  • luxcem/memopol
  • TAlone/memopol
10 résultats
Afficher les modifications
Affichage de
avec 230 ajouts et 695 suppressions
docs/img/score_formula.png

5,99 ko

.. Memopol documentation master file, created by
sphinx-quickstart on Thu Nov 12 22:42:47 2015.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Memopol's documentation!
===================================
Contents:
.. toctree::
:maxdepth: 2
usage
deploy-custom
administration
scores
api
development
hacker
setup_solr
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
Memopol Scores
~~~~~~~~~~~~~~
Score computation
=================
In Memopol, each representative has a score that shows how their votes match
the recommendations made by administrators on the instance. The total score
of a representative is the sum of their score on each dossier, which in turn is
the sum of their score on each proposal with a recommendation on the dossier.
Each recommendation made by administrators has a weight: a number that tells how
important a specific proposal is. When the representative vote on a proposal
matches the recommendation, their score on the proposal is set to +weight.
Otherwise, it is set to -weight.
Score decay parameters
======================
Memopol allows to set decay parameters so that older votes have a lower
importance in the total representative score. By default, those parameters are
set to values that disable the score decay, so that each vote contributes
identically to the total score no matter how old it is.
The formula used to compute score is the following:
.. image:: img/score_formula.png
Where:
* ``baseScore`` is the base score for the vote computed as explained above;
* ``voteAge`` is the age of the vote in days;
* ``decayNum`` and ``decayDenom`` define the decay rate;
* ``exponent`` define the steepness of the decay.
The corresponding parameters can be set from the Memopol administration
interface (Memopol Settings > Settings); settings keys are ``SCORE_DECAY_NUM``,
``SCORE_DECAY_DENOM``, ``SCORE_EXPONENT``. Additionnaly, the ``SCORE_DECIMALS``
parameter sets how many decimal places are visible when scores are displayed.
The default values for those settings disable score decay by setting
``SCORE_DECAY_NUM`` to 0, ``SCORE_DECAY_DENOM`` and ``SCORE_EXPONENT`` to 1.
If you want to use score decay, start by setting ``SCORE_DECAY_NUM`` to 1, and
``SCORE_DECAY_DENOM`` to the number of days you want votes to matter. The graph
below shows how a score of 1.0 will decay with a 1-year decay (the X axis is in
days).
.. image:: img/score_1year.png
Increasing ``SCORE_DECAY_DENOM`` will make votes matter longer. Here is the
same example but with a 10-year decay.
.. image:: img/score_10years.png
Increasing ``SCORE_EXPONENT`` instead will make the decay cutoff steeper. Here
is an example with a 1-year decay and the exponent set to 6.
.. image:: img/score_exp6.png
Increasing it dramatically will create a brutal cutoff; here is the same example
with the exponent set to 1000:
.. image:: img/score_exp1k.png
Setup Solr with Memopol
=======================
Solr is used to perform search in the data. Currently, it is used only for
reprensatative search autocomplete.
This howto will be based on the current stable version of Debian (jessie).
Installation on Debian Stable
-----------------------------
Solr is in the official repository of debian, you can install it launching::
# apt install solr-tomcat/stable
Package: solr-tomcat
Source: lucene-solr
Version: 3.6.2+dfsg-5
Installed-Size: 65,5 kB
Maintainer: Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>
Depends: solr-common (= 3.6.2+dfsg-5), tomcat7
Conflicts: solr-jetty, solr-tomcat6
Homepage: http://lucene.apache.org
Section: java
Priority: optional
Download-Size: 8 598 B
APT-Sources: http://ftp.fr.debian.org/debian/ jessie/main amd64 Packages
Description: Enterprise search server based on Lucene3 - Tomcat integration
Solr is an open source enterprise search server based on the Lucene
Java search library, with XML/HTTP and JSON APIs, hit highlighting,
faceted search, caching, replication, and a web administration
interface. It runs in a Java servlet container such as Tomcat.
.
This package provides the Tomcat integration files for Solr.
By default, the solr server is listen on localhost:8080. Memopol is configured
to use this addess by default, in production. If you install a newer version of
Solr (6.5.1 is the latest release), you should update the settings.py::
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.solr_backend.SolrEngine',
'URL': 'http://127.0.0.1:8080/solr',
},
}
Configure Solr
---------------
You can use the default Solr configuration to use it, You just need to generate
the schema.xml file and copy it in `/etc/solr/`. To generate the schema.xml
file::
$ memopol build_solr_schema > schmema.xml
## Copy the schema.xml file into /etc/solr/conf
$ sudo cp schema.xml /etc/solr/conf/
## Restart tomcat
$ sudo /etc/init.d/tomcat7 restart
Last step, you need to build the index by using::
$ memopol rebuild_index
The solr is now setup and production ready.
Populate data in Solr
---------------------
Django-haystack plugin offers two ways to populate data. The first way is by
using a cron job to update the index, the other way is to use `Dango signals to
update / delete datas <https://django-haystack.readthedocs.io/en/v2.6.0/signal_processors.html>`_.
For now, Haystack is not configured to be used with Django signals. It is
necessary to add a cron job to update the index ::
$ memopol update_index
`More informations about cron with Solr <http://django-haystack.readthedocs.io/en/v2.6.0/searchindex_api.html?highlight=cron#keeping-the-index-fresh>`_
User guide
~~~~~~~~~~
Authentication in the admin backend
===================================
As a content-editor, you should be able to connect to the administration
interface with the credentials and link that your administrator sent you. If
they haven't, please email them with a request and eventually a link to
:doc:`administration`.
Managing representative positions
=================================
Our dear representatives make promises to be elected. These can be submitted by
visitors on the front-end so one of your tasks is to review them and publish
them if they are appropriate.
In "Positions -> Position -> Change" (url should be
``/admin/positions/position/``), above the list table, click the "Published"
select box and choose the "Published: No" option, then click "Search".
Alternatively, you may bookmark
``/admin/positions/position/?published__exact=0``.
Click a "Position" and if it's appropriate then check the "Published" checkbox
and click "Save". The position will now appear in the representative detail
page.
Managing vote recommendations
=============================
A recommendation represents the vote we want representatives to make on a
proposal: representatives doing the "recommended" vote will have their score
increased, others will have their score decreased. Some recommendations may be
more important than others, you can change the number of score points a
recommendation is worth by changing its "weight" (must be a positive integer).
In "Votes -> Recommendations -> Change" (url should be
``/admin/votes/recommendation/``), you can create, update or remove
recommendations.
Your change won't be visible on score as soon as you make it.
Data updates
============
Data updates daily:
- representatives details are updated daily,
- dossiers are synchronised daily,
- proposals are synchronised daily, when proposals are synchronised, they
become available to use in "Recommendations".
- votes and scores are synchronised daily for proposals which have a
"Recommendation",
This means that if a proposal arrives today in the database, you may add a
recommendation for it. The next day, votes will be imported and scores will be
updated as well.
var gulp = require('gulp');
var less = require('gulp-less');
var watch = require('gulp-watch');
var minifycss = require('gulp-minify-css');
var rename = require('gulp-rename');
var gzip = require('gulp-gzip');
var livereload = require('gulp-livereload');
var gzip_options = {
threshold: '1kb',
gzipOptions: {
level: 9
}
};
var less_src = ['static/less/base.less', 'static/less/libs.less'];
/* Compile Our Sass */
gulp.task('less', function() {
return gulp.src(less_src)
.pipe(less())
.pipe(gulp.dest('static/stylesheets'))
.pipe(rename({suffix: '.min'}))
.pipe(minifycss())
.pipe(gulp.dest('static/stylesheets'))
// .pipe(gzip(gzip_options))
// .pipe(gulp.dest('static/stylesheets'))
.pipe(livereload());
});
/* Watch Files For Changes */
gulp.task('watch', function() {
livereload.listen();
gulp.watch('static/less/*.less', ['less']);
/* Trigger a live reload on any Django template changes */
gulp.watch('**/templates/*').on('change', livereload.changed);
});
gulp.task('default', ['less', 'watch']);
# coding: utf-8
# This file is part of mempol.
#
# mempol is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of
# the License, or any later version.
#
# mempol is distributed in the hope that it will
# be useful, but WITHOUT ANY WARRANTY; without even the implied
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# See the GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Affero Public
# License along with django-representatives.
# If not, see <http://www.gnu.org/licenses/>.
#
# Copyright (C) 2015 Arnaud Fabre <af@laquadrature.net>
from __future__ import absolute_import
from django.shortcuts import redirect
from .tasks import representatives_update_all as rpr_update_task
def representatives_update_all(request):
rpr_update_task.delay()
return redirect('/admin')
import pyprind
from django.core.management.base import BaseCommand
from legislature.models import MemopolRepresentative
class Command(BaseCommand):
def handle(self, *args, **options):
bar = pyprind.ProgBar(MemopolRepresentative.objects.all().count())
for representative in MemopolRepresentative.objects.all():
representative.update_score()
bar.update(item_id = str(representative))
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
class Migration(migrations.Migration):
dependencies = [
('representatives', '0003_auto_20150702_1827'),
]
operations = [
migrations.CreateModel(
name='MemopolRepresentative',
fields=[
('representative_ptr', models.OneToOneField(parent_link=True, auto_created=True, primary_key=True, serialize=False, to='representatives.Representative')),
('score', models.IntegerField(default=0)),
('country', models.ForeignKey(to='representatives.Country', null=True)),
],
options={
'abstract': False,
},
bases=('representatives.representative',),
),
]
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
class Migration(migrations.Migration):
dependencies = [
('representatives', '0004_auto_20150709_1601'),
('legislature', '0001_initial'),
]
operations = [
migrations.AddField(
model_name='memopolrepresentative',
name='main_mandate',
field=models.ForeignKey(default=True, to='representatives.Mandate', null=True),
),
]
# coding: utf-8
# This file is part of memopol.
#
# memopol is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of
# the License, or any later version.
#
# memopol is distributed in the hope that it will
# be useful, but WITHOUT ANY WARRANTY; without even the implied
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# See the GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Affero Public
# License along with django-representatives.
# If not, see <http://www.gnu.org/licenses/>.
#
# Copyright (C) 2015 Arnaud Fabre <af@laquadrature.net>
from datetime import datetime
from django.db import models
from django.core.exceptions import ObjectDoesNotExist
from django.db.models.signals import post_save
from django.dispatch import receiver
# from django.utils.functional import cached_property
from representatives.models import Representative, Mandate, Country
from votes.models import MemopolVote
from core.utils import create_child_instance_from_parent
class MemopolRepresentative(Representative):
country = models.ForeignKey(Country, null=True)
score = models.IntegerField(default=0)
main_mandate = models.ForeignKey(Mandate, null=True, default=True)
def update_score(self):
score = 0
for vote in self.votes.all():
score += vote.absolute_score
self.score = score
self.save()
def update_country(self):
# Create a country if it does'nt exist
# The representative's country is the one associated
# with the last 'country' mandate
try:
country_mandate = self.mandates.filter(
group__kind='country'
).order_by('-begin_date')[0:1].get()
country, _ = Country.objects.get_or_create(
name=country_mandate.group.name,
code=country_mandate.group.abbreviation
)
self.country = country
except ObjectDoesNotExist:
self.country = None
self.save()
def update_main_mandate(self):
try:
self.main_mandate = self.mandates.get(
end_date__gte=datetime.now(),
group__kind='group'
)
except Mandate.DoesNotExist:
self.main_mandate = None
self.save()
def update_all(self):
self.update_country()
self.update_score()
self.update_main_mandate()
def active_mandates(self):
return self.mandates.filter(
end_date__gte=datetime.now()
)
def former_mandates(self):
return self.mandates.filter(
end_date__lte=datetime.now()
)
def votes_with_proposal(self):
return MemopolVote.objects.select_related(
'proposal',
'proposal__recommendation'
).filter(representative=self)
@receiver(post_save, sender=Representative)
def create_memopolrepresentative_from_representative(instance, **kwargs):
memopol_representative = create_child_instance_from_parent(MemopolRepresentative, instance)
memopol_representative.save()
# coding: utf-8
# This file is part of memopol.
#
# memopol is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of
# the License, or any later version.
#
# memopol is distributed in the hope that it will
# be useful, but WITHOUT ANY WARRANTY; without even the implied
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# See the GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Affero Public
# License along with django-representatives.
# If not, see <http://www.gnu.org/licenses/>.
#
# Copyright (C) 2015 Arnaud Fabre <af@laquadrature.net>
from datetime import datetime
from django.http import Http404
from django.shortcuts import render, get_object_or_404
from django.core.paginator import Paginator, EmptyPage, PageNotAnInteger
from django.db.models import Q
from representatives.models import Group
from legislature.models import MemopolRepresentative
def retrieve(request, pk=None, name=None):
if pk:
representative = get_object_or_404(
MemopolRepresentative,
id=pk
)
elif name:
representative = get_object_or_404(
MemopolRepresentative,
full_name=name
)
else:
return Http404()
return render(
request,
'legislature/representative_view.html',
{'representative': representative}
)
def representatives_by_group(request, group_kind, group_abbr=None,
group_name=None, search=None, group_id=None):
if group_id:
representative_list = MemopolRepresentative.objects.filter(
mandates__group_id=group_id,
mandates__end_date__gte=datetime.now()
)
elif group_abbr:
representative_list = MemopolRepresentative.objects.filter(
mandates__group__abbreviation=group_abbr,
mandates__group__kind=group_kind,
mandates__end_date__gte = datetime.now()
)
elif group_name:
representative_list = MemopolRepresentative.objects.filter(
Q(mandates__group__name__icontains=group_name),
mandates__group__kind=group_kind,
mandates__end_date__gte = datetime.now()
)
elif search:
try:
Group.objects.get(abbreviation=search, kind=group_kind)
representative_list = MemopolRepresentative.objects.filter(
mandates__group__abbreviation=search,
mandates__group__kind=group_kind,
mandates__end_date__gte = datetime.now()
)
except Group.DoesNotExist:
representative_list = MemopolRepresentative.objects.filter(
Q(mandates__group__abbreviation__icontains=search) |
Q(mandates__group__name__icontains=search),
mandates__group__kind=group_kind,
mandates__end_date__gte = datetime.now()
)
# Select distinct representatives and filter by search
representative_list = list(set(
_filter_by_search(request, representative_list)
))
return _render_list(request, representative_list)
def _filter_by_search(request, representative_list):
"""
Return a representative_list filtered by
the representative name provided in search form
"""
search = request.GET.get('search')
if search:
return representative_list.filter(
Q(full_name__icontains=search)
)
else:
return representative_list
def _render_list(request, representative_list, num_by_page=30):
"""
Render a paginated list of representatives
"""
paginator = Paginator(representative_list, num_by_page)
page = request.GET.get('page')
try:
representatives = paginator.page(page)
except PageNotAnInteger:
representatives = paginator.page(1)
except EmptyPage:
representatives = paginator.page(paginator.num_pages)
context = {}
queries_without_page = request.GET.copy()
if 'page' in queries_without_page:
del queries_without_page['page']
context['queries'] = queries_without_page
context['representatives'] = representatives
context['representative_num'] = paginator.count
return render(
request,
'legislature/representative_list.html',
context
)
def groups_by_kind(request, kind):
groups = Group.objects.filter(
kind=kind,
mandates__end_date__gte=datetime.now()
).distinct().order_by('name')
return render(
request,
'legislature/groups_list.html',
{'groups': groups}
)
# coding: utf-8
# This file is part of memopol.
#
# memopol is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of
# the License, or any later version.
#
# memopol is distributed in the hope that it will
# be useful, but WITHOUT ANY WARRANTY; without even the implied
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# See the GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Affero Public
# License along with django-representatives.
# If not, see <http://www.gnu.org/licenses/>.
#
# Copyright (C) 2015 Arnaud Fabre <af@laquadrature.net>
from __future__ import absolute_import
from celery import shared_task
from .models import MemopolRepresentative
@shared_task
def representatives_update_all():
'''
Call MemopolRepresentative.update_all methods
'''
for representative in MemopolRepresentative.objects.all():
representative.update_all()
- load representatives_extras
- load humanize
.representative
%h1.name
={representative.full_name}
.row
.col-md-4
%p.photo
%img{:src => "{{ representative.photo }}"}/
.col-md-8
%table.table.table-condensed.detail-view
%tr.score
%th Score
%td
= representative.score|score_label
%tr
%th Country
%td
= representative.country|country_flag
%tr
%th Party
%td
%a{:href => "{{ representative.main_mandate|by_group_url }}"}
{{ representative.main_mandate.role }} of
{{ representative.main_mandate.group.name }}
%tr
%th Biography
%td
Born in {{ representative.birth_place }} the
{{ representative.birth_date|naturalday:'d/m/Y' }}
({{ representative.get_gender_display }})
-# List representatives' groups
- extends 'base.html'
- load representatives_extras
- block content
%table.table
- for group in groups
%tr
%td
%a{'href': '{{ group | by_group_url }}'}w
- if group.abbreviation
={group.abbreviation}
%td
%a{'href': '{{ group | by_group_url }}'}= group.name
-# List representatives
- extends 'base.html'
- load humanize
- load representatives_extras
- load bootstrap3
- block head
{{ position_form.media }}
- block content
.row
.col-md-6
- include 'legislature/blocks/representative_block.html' with representative=representative
.col-md-6
%h2#votes Votes
%table.table.table-condensed.votes
%tr
%th Title
%th.icon-cell
Memopol recommendation
%th.icon-cell
Representative vote
%th.icon-cell
Score
- for vote in representative.votes_with_proposal.all
%tr
%td= vote.proposal.recommendation.title
%td.icon-cell
= vote.proposal.recommendation.recommendation|position_icon
%td.icon-cell
= vote.position|position_icon
%td.icon-cell
= vote.absolute_score|score_label
%h2 Mandates
%table.table.table-condensed.mandates
- for mandate in representative.active_mandates
%tr.mandate
%td= mandate.role
%td
%a{:href => "{{ mandate|by_group_url }}"}
{{ mandate.group.name }} ({{ mandate.group.abbreviation }})
%td= mandate.begin_date|mandate_date:'d/m/Y'
%td= mandate.end_date|mandate_date:'d/m/Y'
%td= mandate.constituency.name
.positions
%h2 Public positions
%table.table.table-condensed
- for position in representative.positions.published.all
%tr.position
%td= position.datetime|naturalday:'d/m/Y'
%td
%a{:href => '{% url "positions:position-detail" position.pk %}'}
=position.text|truncatewords:8
%td
= position.dossier
%td
%a{:href => '{{position.link}}'}
= position.link
%form{:action => '{% url "positions:position-create" %}',
:method => 'post'}
- csrf_token
%input{:type => 'hidden',
:name => 'representative',
:value => '{{ representative.id }}'}
- bootstrap_form position_form
- buttons
%button{'type': 'submit', 'class': 'btn btn-primary'}
{% bootstrap_icon "star" %} Submit
- endbuttons
- extends 'base.html'
- load representatives_extras
- block content
- include 'legislature/search.html'
- include "core/blocks/pagination.html"
%table.table
%tr
%th
Photo
%th
Name
%th
Country
%th
Group
%th
Score
- for representative in object_list
%tr
%td
%a{'href': "{% url 'legislature:representative-detail' name=representative.slug %}"}
%img{'src': '={representative.photo}', 'width': '80'}/
%td
%a{'href': "{% url 'legislature:representative-detail' name=representative.slug %}"}
= representative.full_name
%td
%a{'href': "{% url 'legislature:representative-index' group_kind='country' group=representative.country.code %}"}
= representative.country|country_flag
%td
%a{'href': "{{ representative.main_mandate|by_group_url }}"}
= representative.main_mandate.group.abbreviation
%td
= representative.score|score_label
- include "core/blocks/pagination.html"
<form action="" method="get">
{% csrf_token %}
<label for="search">Search : </label>
<input id="search" type="text" name="search">
<input type="submit" value="Go">
</form>
# coding: utf-8
# This file is part of memopol.
#
# memopol is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of
# the License, or any later version.
#
# memopol is distributed in the hope that it will
# be useful, but WITHOUT ANY WARRANTY; without even the implied
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# See the GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Affero Public
# License along with django-representatives.
# If not, see <http://www.gnu.org/licenses/>.
#
# Copyright (C) 2015 Arnaud Fabre <af@laquadrature.net>
"""
This file contains all templatetags used by the representative app
"""
from django import template
from django.core.urlresolvers import reverse
from django.utils.safestring import mark_safe
from django.contrib.humanize.templatetags.humanize import naturalday
from representatives.models import Mandate, Group
register = template.Library()
@register.filter
def mandate_date(date, arg=None):
if date.year == 9999:
return 'present'
else:
return naturalday(date, arg)
@register.filter
def position_icon(position):
if position == 'for':
return mark_safe(
'<i \
aria-label="for" \
class="fa fa-thumbs-up vote_positive" \
title="for" \
></i>')
elif position == 'against':
return mark_safe(
'<i \
aria-label="against" \
class="fa fa-thumbs-down vote_negative" \
title="against" \
></i>')
else:
return mark_safe(
'<i \
aria-label="abstain" \
class="fa fa-circle-o vote_abstain" \
title="abstain" \
></i>')
@register.filter
def score_label(score):
if score > 0:
return mark_safe('<span class="label label-success">{}</span>'.format(score))
elif score < 0:
return mark_safe('<span class="label label-danger">{}</span>'.format(score))
else:
return mark_safe('<span class="label label-default">{}</span>'.format(score))
@register.filter
def country_flag(country):
return mark_safe('<span class="flag-icon flag-icon-{code}"></span> {name}'.format(
name=country.name,
code=country.code.lower()
))
@register.filter
def by_group_url(group):
if isinstance(group, Mandate):
group = group.group
if not isinstance(group, Group):
return ''
kwargs = {'group_kind': group.kind}
if group.abbreviation:
kwargs['group'] = group.abbreviation
else:
kwargs['group'] = group.name
# kwargs['group_id'] = group.id
return reverse(
'legislature:representative-index',
kwargs=kwargs
)
from django.test import TestCase
# Create your tests here.