Update README.rst

fix CI badge and remove the coverage one
2024-03-13 01:13:02 +01:00 · 2024-03-13 00:57:26 +01:00 · 2024-03-13 00:35:48 +01:00 · 2024-03-13 00:29:50 +01:00 · 2024-03-13 00:21:04 +01:00 · 2024-03-13 00:18:47 +01:00
73 changed files with 1435 additions and 807 deletions
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@ -0,0 +1,19 @@
+name: CI
+
+on:
+  push:
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  build_and_test:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v2
+
+    - name: Install packages and setup ldap/AD
+      run: sudo ./tests/test_env/deploy.sh
+
+    - name: Test
+      run: python3 setup.py test 
--- a/.gitignore
+++ b/.gitignore
@ -54,3 +54,4 @@ coverage.xml
 # Sphinx documentation
 docs/_build/

+ldapcherry-dev.ini
--- a/.travis.yml
+++ b/.travis.yml
@ -1,31 +1,40 @@
 sudo: required
-dist: trusty
+dist: xenial
 language: python

-#env:
-#  - TRAVIS="yes"
-
 before_install:
  - '[ "$TEST_PEP8" == "1" ] || sudo ./tests/test_env/deploy.sh'

-python:
-  - "2.7"
-
 install: 
-  - pip install -e .
-  - "if [[ $TEST_PEP8 == '1' ]]; then pip install pep8; fi"
+  - "pip install -e . -r $REQ_FILE"
+  - "if [[ $TEST_PEP8 == '1' ]]; then pip install pycodestyle; fi"
  - pip install passlib
  - pip install coveralls

-# command to run tests
-#
-#script: 
-#  - coverage run --source=ldapcherry setup.py test
-script: "if [[ $TEST_PEP8 == '1' ]]; then pep8 --repeat --show-source --exclude=.venv,.tox,dist,docs,build,*.egg,tests,misc . scripts/ldapcherryd; else coverage run --source=ldapcherry setup.py test; fi"
+script: "if [[ $TEST_PEP8 == '1' ]]; then pycodestyle --repeat --show-source --exclude=.venv,.tox,dist,docs,build,*.egg,tests,misc,setup.py .; else coverage run --source=ldapcherry setup.py test; fi"
 matrix:
  include:
    - python: "2.7"
-      env: TEST_PEP8=1
+      env:
+        TEST_PEP8=1
+        REQ_FILE=requirements.txt
+    - python: "2.7"
+      env:
+        TEST_PEP8=0
+        REQ_FILE=requirements-el7.txt
+    - python: "2.7"
+      env:
+        TEST_PEP8=0
+        REQ_FILE=requirements-stretch.txt
+    - python: "2.7"
+      env:
+        TEST_PEP8=0
+        REQ_FILE=requirements.txt
+    - python: "3.6"
+      env:
+        TEST_PEP8=0
+        REQ_FILE=requirements.txt
+
 after_success:
  - coveralls
 after_failure:
--- a/ChangeLog.rst
+++ b/ChangeLog.rst
@ -1,6 +1,49 @@
 Dev
 ***

+Version 1.1.1
+*************
+
+* [fix ] fix double escaping issues introduced in 1.0.0
+* [fix ] fix missing url escaping in links with querystring parameters (delete and modify page mostly)
+* [fix ] fix log level not being honored in the backends
+* [impr] clarify the role of 'key: True' of attributes.yml in the documentation
+* [impr] add a few more comments in the .ini file to explain better the \*_filter_tmpl and group_attr parameters
+* [impr] add debug log to help debug ldap search filters
+
+Version 1.1.0
+*************
+
+* [feat] add stdout as a valid log method (useful when running with docker)
+
+Version 1.0.1
+*************
+
+* [fix ] fix error handling when adding user that already exists
+
+Version 1.0.0
+*************
+
+* [sec ] fix XSS injection in the url redirect in the login page (thanks to jthiltges)
+* [fix ] fix configuration consistency check for attribute file (error if a given backend is not declared in main .ini file but in attributes)
+* [fix ] remove a few deprecation warnings
+* [fix ] fix potential issue with group names containing non-ascii characters
+* [feat] support for python 3
+* [feat] support for python-ldap 3.X.X
+* [impr] better log error message if inconsistency between role, attribute and main .ini file for backends
+* [impr] more systematic use of html and url escaping in the html rendering to prevent against content injection (thanks to jthiltges)
+* [impr] more testing for various versions of python and python-ldap
+
+Version 0.5.2
+*************
+
+* [fix ] regression in 0.5.1, setup.py could not work without the dependencies already installed
+
+Version 0.5.1
+*************
+
+* [impr] cleaner align of labels (input-group-addon width)
+
 Version 0.5.0
 *************

--- a/README.rst
+++ b/README.rst
@ -6,11 +6,9 @@

 Nice and simple application to manage users and groups in multiple directory services.

-.. image:: https://travis-ci.org/kakwa/ldapcherry.svg?branch=master
-    :target: https://travis-ci.org/kakwa/ldapcherry
-    
-.. image:: https://coveralls.io/repos/kakwa/ldapcherry/badge.svg 
-    :target: https://coveralls.io/r/kakwa/ldapcherry
+.. image:: https://github.com/kakwa/ldapcherry/actions/workflows/tests.yml/badge.svg
+    :target: https://github.com/kakwa/ldapcherry/actions/workflows/tests.yml
+    :alt: CI
    
 .. image:: https://img.shields.io/pypi/v/ldapcherry.svg
    :target: https://pypi.python.org/pypi/ldapcherry
@ -36,7 +34,7 @@ Nice and simple application to manage users and groups in multiple directory ser

 LdapCherry is a CherryPY application to manage users and groups in multiple directory services.

-It's main features are:
+Its main features are:

 * manage multiple directories/databases backends in an unified way
 * roles management (as in "groups of groups")
@ -66,11 +64,16 @@ The default backend plugins permit to manage Ldap and Active Directory.

 .. sourcecode:: bash

+    # clone the repository
+    $ git clone https://github.com/kakwa/ldapcherry && cd ldapcherry
+
    # change the directory where to put the configuration (default: /etc)
-    $ export SYSCONFDIR=<sys conf dir>
+    $ export SYSCONFDIR=/etc
+    # change the directory where to put the resource (default: /usr/share)
+    $ export DATAROOTDIR=/usr/share/
    
    # install ldapcherry
-    $ pip install ldapcherry
+    $ python setup.py install

    # edit configuration files
    $ vi /etc/ldapcherry/ldapcherry.ini
@ -78,7 +81,7 @@ The default backend plugins permit to manage Ldap and Active Directory.
    $ vi /etc/ldapcherry/attributes.yml

    # launch ldapcherry
-    $ ldapcherryd -c /etc/ldapcherry/ldapcherry.ini
+    $ ldapcherryd -c /etc/ldapcherry/ldapcherry.ini -D


 ***********
@ -91,7 +94,7 @@ LdapCherry is published under the MIT Public License.
  Discussion / Help / Updates
 *******************************

-* IRC: `Freenode <http://freenode.net/>`_ ``#ldapcherry`` channel
+* IRC: `Libera <https://libera.chat/>`_ ``#ldapcherry`` channel
 * Bugtracker: `Github <https://github.com/kakwa/ldapcherry/issues>`_

 ----
--- a/conf/ldapcherry.ini
+++ b/conf/ldapcherry.ini
@ -5,6 +5,10 @@
 server.socket_host = '127.0.0.1'
 # port
 server.socket_port = 8080
+
+# it's also possible to run bound to a unix socket
+#server.socket_file = '/tmp/lc.sock'
+
 # number of threads
 server.thread_pool = 8
 #don't show traceback on error
@ -24,6 +28,14 @@ request.show_tracebacks = False
 ## error and ldapcherry log file
 #log.error_file = '/tmp/ldapcherry_error.log'

+#####################################
+#  configuration to log to stdout   #
+#####################################
+## logger stdout for access log
+#log.access_handler = 'stdout'
+## logger stdout for error and ldapcherry log
+#log.error_handler = 'stdout'
+
 #####################################
 #  configuration to log in syslog   #
 #####################################
@ -94,16 +106,24 @@ ldap.timeout = 1
 ldap.groupdn = 'ou=group,dc=example,dc=org'
 # users dn
 ldap.userdn = 'ou=people,dc=example,dc=org'
-# ldapsearch filter to get a user
+
+# ldapsearch filter to get one specific user
+# %(username)s is content of the attribute marked 'key: True' in the attributes.file config file
 ldap.user_filter_tmpl = '(uid=%(username)s)'
 # ldapsearch filter to get groups of a user
+# %(username)s is content of the attribute marked 'key: True' in the attributes.file config file
 ldap.group_filter_tmpl = '(member=uid=%(username)s,ou=People,dc=example,dc=org)'
 # filter to search users
+# %(searchstring)s is the content passed through the search box
 ldap.search_filter_tmpl = '(|(uid=%(searchstring)s*)(sn=%(searchstring)s*))'

 # ldap group attributes and how to fill them
+# 'member' is the name of the attribute
+# for the template, any of the user's ldap attributes can be user
 ldap.group_attr.member = "%(dn)s"
+# same with memverUid and the uid user's attribute
 #ldap.group_attr.memberUid = "%(uid)s"
+
 # object classes of a user entry
 ldap.objectclasses = 'top, person, posixAccount, inetOrgPerson'
 # dn entry attribute for an ldap user
--- a/conf/roles.yml
+++ b/conf/roles.yml
@ -25,12 +25,12 @@ admin-lv2:
 #        ad:
 #            - Administrators

-developpers:
+developers:
    display_name: Developpers
    description: Developpers of the system
    backends_groups:
        ldap:
-            - cn=developpers,ou=Group,dc=example,dc=org
+            - cn=developers,ou=Group,dc=example,dc=org
            - cn=users,ou=Group,dc=example,dc=org

 users:
--- a/docs/backends.rst
+++ b/docs/backends.rst
@ -47,48 +47,55 @@ Configuration

 The ldap backend exposes the following parameters:

-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-|      Parameter           | Section  |            Description             |           Values         |                Comment                     |
-+==========================+==========+====================================+==========================+============================================+
-| uri                      | backends | The ldap uri to access             | ldap uri                 | * use ldap:// for clear/starttls           |
-|                          |          |                                    |                          | * use ldaps:// for ssl                     |
-|                          |          |                                    |                          | * custom port: ldap://<host>:<port>        |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| ca                       | backends | Path to the CA file                | file path                | optional                                   |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| starttls                 | backends | Use starttls                       | 'on' or 'off'            | optional                                   |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| checkcert                | backends | Check the server certificat        | 'on' or 'off'            | optional                                   |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| binddn                   | backends | The bind dn to use                 | ldap dn                  | This dn must have read/write permissions   |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| password                 | backends | The password of the bind dn        | password                 |                                            |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| timeout                  | backends | Ldap connexion timeout             | integer (second)         |                                            |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| password                 | backends | The password of the bind dn        | password                 |                                            |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| groupdn                  | backends | The ldap dn where groups are       | ldap dn                  |                                            |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| userdn                   | backends | The ldap dn where users are        | ldap dn                  |                                            |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| user_filter_tmpl         | backends | The search filter template         | ldap search filter       | The user identifier is passed through      |
-|                          |          | to recover a given user            | template                 | the **username** variable (*%(username)s*).|
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| group_filter_tmpl        | backends | The search filter template to      | ldap search filter       | The following variables are usable:        |
-|                          |          | recover the groups of a given user | template                 | * **username**: the user key attribute     |
-|                          |          |                                    |                          | * **userdn**: the user ldap dn             |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| group_attr.<member attr> | backends | Member attribute template value    | template                 | * <member attr> is the member attribute    |
-|                          |          |                                    |                          |   in groups dn entries                     |
-|                          |          |                                    |                          | * every user attributes are exposed        |
-|                          |          |                                    |                          |   in the template                          |
-|                          |          |                                    |                          | * multiple attributes can be set           |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| objectclasses            | backends | list of object classes for users   | comma separated list     |                                            |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
-| dn_user_attr             | backends | attribute used in users dn         | dn attribute             |                                            |
-+--------------------------+----------+------------------------------------+--------------------------+--------------------------------------------+
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+|      Parameter           | Section  |            Description             |           Values         |                Comment                         |
+==========================+==========+====================================+==========================+================================================+
+| uri                      | backends | The ldap uri to access             | ldap uri                 | * use ldap:// for clear/starttls               |
+|                          |          |                                    |                          | * use ldaps:// for ssl                         |
+|                          |          |                                    |                          | * custom port: ldap://<host>:<port>            |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| ca                       | backends | Path to the CA file                | file path                | optional                                       |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| starttls                 | backends | Use starttls                       | 'on' or 'off'            | optional                                       |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| checkcert                | backends | Check the server certificat        | 'on' or 'off'            | optional                                       |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| binddn                   | backends | The bind dn to use                 | ldap dn                  | This dn must have read/write permissions       |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| password                 | backends | The password of the bind dn        | password                 |                                                |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| timeout                  | backends | Ldap connexion timeout             | integer (second)         |                                                |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| password                 | backends | The password of the bind dn        | password                 |                                                |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| groupdn                  | backends | The ldap dn where groups are       | ldap dn                  |                                                |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| userdn                   | backends | The ldap dn where users are        | ldap dn                  |                                                |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| user_filter_tmpl         | backends | The search filter template         | ldap search filter       | The user identifier is passed through          |
+|                          |          | to recover a given user            | template                 | the **username** variable (*%(username)s*)     |
+|                          |          |                                    |                          |                                                |
+|                          |          |                                    |                          | **username** is the content of the             |
+|                          |          |                                    |                          | the attribute marked by '**key: True**'        |
+|                          |          |                                    |                          | in the **attributes.yml** file                 |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| group_filter_tmpl        | backends | The search filter template to      | ldap search filter       | The following variables are usable:            |
+|                          |          | recover the groups of a given user | template                 |                                                |
+|                          |          | recover the groups of a given user | template                 | * **username**: the user's key attribute       |
+|                          |          |                                    |                          | * **userdn**: the user's ldap dn               |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| group_attr.<member attr> | backends | Member attribute template value    | template                 | * <member attr> is the member attribute        |
+|                          |          |                                    |                          |   in groups dn entries                         |
+|                          |          |                                    |                          | * every user attributes are exposed            |
+|                          |          |                                    |                          |   in the template                              |
+|                          |          |                                    |                          | * multiple <memver attr> attributes            |
+|                          |          |                                    |                          |   can be set (ex: group_attr.member            |
+|                          |          |                                    |                          |   (ex: group_attr.member, group_attr.usermemb) |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| objectclasses            | backends | list of object classes for users   | comma separated list     |                                                |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+
+| dn_user_attr             | backends | attribute used in users dn         | dn attribute             |                                                |
+--------------------------+----------+------------------------------------+--------------------------+------------------------------------------------+


 Example
@ -96,46 +103,58 @@ Example

 .. sourcecode:: ini

-    [backends]
+   [backends]
    
-    # name of the module
-    ldap.module = 'ldapcherry.backend.backendLdap'
-    # display name of the ldap
-    ldap.display_name = 'My Ldap Directory'
-    
-    # uri of the ldap directory
-    ldap.uri = 'ldap://ldap.ldapcherry.org'
-    # ca to use for ssl/tls connexion
-    #ldap.ca = '/etc/dnscherry/TEST-cacert.pem'
-    # use start tls
-    #ldap.starttls = 'off'
-    # check server certificate (for tls)
-    #ldap.checkcert = 'off'
-    # bind dn to the ldap
-    ldap.binddn = 'cn=dnscherry,dc=example,dc=org'
-    # password of the bind dn
-    ldap.password = 'password'
-    # timeout of ldap connexion (in second)
-    ldap.timeout = 1
-    
-    # groups dn
-    ldap.groupdn = 'ou=group,dc=example,dc=org'
-    # users dn
-    ldap.userdn = 'ou=people,dc=example,dc=org'
-    # ldapsearch filter to get a user
-    ldap.user_filter_tmpl = '(uid=%(username)s)'
-    # ldapsearch filter to get groups of a user
-    ldap.group_filter_tmpl = '(member=uid=%(username)s,ou=People,dc=example,dc=org)'
-    # filter to search users
-    ldap.search_filter_tmpl = '(|(uid=%(searchstring)s*)(sn=%(searchstring)s*))'
-    
-    # ldap group attributes and how to fill them
-    ldap.group_attr.member = "%(dn)s"
-    #ldap.group_attr.memberUid = "%(uid)s"
-    # object classes of a user entry
-    ldap.objectclasses = 'top, person, posixAccount, inetOrgPerson'
-    # dn entry attribute for an ldap user
-    ldap.dn_user_attr = 'uid'
+   #####################################
+   #   configuration of ldap backend   #
+   #####################################
+   
+   # name of the module
+   ldap.module = 'ldapcherry.backend.backendLdap'
+   # display name of the ldap
+   ldap.display_name = 'My Ldap Directory'
+   
+   # uri of the ldap directory
+   ldap.uri = 'ldap://ldap.ldapcherry.org'
+   # ca to use for ssl/tls connexion
+   #ldap.ca = '/etc/dnscherry/TEST-cacert.pem'
+   # use start tls
+   #ldap.starttls = 'off'
+   # check server certificate (for tls)
+   #ldap.checkcert = 'off'
+   # bind dn to the ldap
+   ldap.binddn = 'cn=dnscherry,dc=example,dc=org'
+   # password of the bind dn
+   ldap.password = 'password'
+   # timeout of ldap connexion (in second)
+   ldap.timeout = 1
+   
+   # groups dn
+   ldap.groupdn = 'ou=group,dc=example,dc=org'
+   # users dn
+   ldap.userdn = 'ou=people,dc=example,dc=org'
+   
+   # ldapsearch filter to get one specific user
+   # %(username)s is content of the attribute marked 'key: True' in the attributes.file config file
+   ldap.user_filter_tmpl = '(uid=%(username)s)'
+   # ldapsearch filter to get groups of a user
+   # %(username)s is content of the attribute marked 'key: True' in the attributes.file config file
+   ldap.group_filter_tmpl = '(member=uid=%(username)s,ou=People,dc=example,dc=org)'
+   # filter to search users
+   # %(searchstring)s is the content passed through the search box
+   ldap.search_filter_tmpl = '(|(uid=%(searchstring)s*)(sn=%(searchstring)s*))'
+   
+   # ldap group attributes and how to fill them
+   # 'member' is the name of the attribute
+   # for the template, any of the user's ldap attributes can be user
+   ldap.group_attr.member = "%(dn)s"
+   # same with memverUid and the uid user's attribute
+   #ldap.group_attr.memberUid = "%(uid)s"
+   
+   # object classes of a user entry
+   ldap.objectclasses = 'top, person, posixAccount, inetOrgPerson'
+   # dn entry attribute for an ldap user
+   ldap.dn_user_attr = 'uid'


 Active Directory Backend
--- a/docs/conf.py
+++ b/docs/conf.py
@ -16,7 +16,6 @@
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 import os
 import sys
-import sys

 try:
    from mock import Mock as MagicMock
@ -32,6 +31,9 @@ MOCK_MODULES = ['ldap']
 sys.modules.update((mod_name, Mock()) for mod_name in MOCK_MODULES)

 sys.path.insert(0, os.path.abspath('..'))
+sys.path.insert(0, os.path.abspath('../ldapcherry'))
+
+from version import version

 # -- General configuration -----------------------------------------------------

@ -64,7 +66,7 @@ copyright = u'2016, Pierre-Francois Carpentier'
 #
 # The short X.Y version.
 # The full version, including alpha/beta/rc tags.
-release = '0.5.0'
+release = version

 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
--- a/docs/deploy.rst
+++ b/docs/deploy.rst
@ -112,7 +112,12 @@ If **type** is set to **stringlist** the parameter **values** must be filled wit
 Key attribute:
 ^^^^^^^^^^^^^^

-One attribute must be used as a unique key across all backends:
+One attribute must be used as a unique key across all backends.
+
+It acts as a reconciliation key.
+
+It also marks which attribute must be used within ldapcherry (ex: querysting parameter in links)
+to point to one given user.

 To set the key attribute, you must set **key** to **True** on this attribute.

@ -445,16 +450,16 @@ Logging

 LdapCherry has two loggers, one for errors and applicative actions (login, del/add, logout...) and one for access logs.

-Each logger can be configured to log to syslog, file or be disabled. 
+Each logger can be configured to log to **syslog**, **file**, **stdout** or be disabled.

 Logging parameters:

 +--------------------+---------+---------------------------------+-------------------------------------------------+----------------------------------------+
 |      Parameter     | Section |           Description           |                      Values                     |                 Comment                |
 +====================+=========+=================================+=================================================+========================================+
-| log.access_handler |  global |    Logger type for access log   |            'syslog', 'file', 'none'             |                                        |
+| log.access_handler |  global |    Logger type for access log   |      'syslog', 'file', 'stdout', 'none'         |                                        |
 +--------------------+---------+---------------------------------+-------------------------------------------------+----------------------------------------+
-|  log.error_handler |  global | Logger type for applicative log |             'syslog', 'file', 'none'            |                                        |
+|  log.error_handler |  global | Logger type for applicative log |      'syslog', 'file', 'stdout', 'none'         |                                        |
 +--------------------+---------+---------------------------------+-------------------------------------------------+----------------------------------------+
 |   log.access_file  |  global |     log file for access log     |                 path to log file                | only used if log.access_handler='file' |
 +--------------------+---------+---------------------------------+-------------------------------------------------+----------------------------------------+
@ -477,6 +482,14 @@ Example:
    log.level = 'info'


+.. warning::
+
+   'debug' should not be used in production.
+
+   It tends to log a lot.
+   More significantly can represent a security issue,
+   as things like passwords will be logged 'clear text'.
+
 Custom javascript
 ~~~~~~~~~~~~~~~~~

--- a/docs/goodies.rst
+++ b/docs/goodies.rst
@ -18,17 +18,49 @@ This init script is available in **goodies/init-debian**.
 Apache Vhost
 ------------

+Basic Apache Vhost:
+
 .. literalinclude:: ../goodies/apache.conf
    :language: xml

 Nginx Vhost
 -----------

+Basic Nginx Vhost:
+
 .. literalinclude:: ../goodies/nginx.conf
    :language: yaml

+Nginx Vhost (FastCGI)
+---------------------
+
+Nginx Vhost in FastCGI mode:
+
+.. literalinclude:: ../goodies/nginx-fastcgi.conf
+    :language: yaml
+
+.. warning::
+
+    LdapCherry requires the python flup module to run in FastCGI
+
 Lighttpd Vhost
 --------------

+Basic Lighttpd Vhost
+
 .. literalinclude:: ../goodies/lighttpd.conf
    :language: yaml
+
+Demo Backend Configuration Files
+--------------------------------
+
+The files here are the ones that are used at the demo site at `ldapcherry.kakwalab.ovh <https://ldapcherry.kakwalab.ovh/>`_ and can be used for a self-hosted demo backend:
+
+.. literalinclude:: ../goodies/demo_backend_configs/attributes.yml
+    :language: yaml
+
+.. literalinclude:: ../goodies/demo_backend_configs/roles.yml
+    :language: yaml
+
+.. literalinclude:: ../goodies/demo_backend_configs/ldapcherry.ini
+    :language: ini
--- a/docs/install.rst
+++ b/docs/install.rst
@ -12,18 +12,13 @@ Download the latest release from `GitHub <https://github.com/kakwa/ldapcherry/re
    $ cd ldapcherry*
    $ python setup.py install

-From Pypi
---------
-
-.. sourcecode:: bash 
-
-    $ pip install ldapcherry
-
-or
+Alternatively, you can install from git:

 .. sourcecode:: bash

-    $ easy_install ldapcherry 
+    $ git clone https://github.com/kakwa/ldapcherry 
+    $ cd ldapcherry
+    $ python setup.py install

 Installed files
 ---------------
@ -38,8 +33,12 @@ These directories can be changed by exporting the following variables before lau

 .. sourcecode:: bash

-    #optional, default sys.prefix + 'share' (/usr/share/ on most Linux)
+    # optional, default sys.prefix + 'share' (/usr/share/ on most Linux)
    $ export DATAROOTDIR=/usr/local/share/
-    #optional, default /etc/
+
+    # optional, default /etc/
    $ export SYSCONFDIR=/usr/local/etc/ 

+.. note:: if --root is passed, the install prefix is honored for these directories
+
+.. warning:: If you change these directories, **templates.dir** and **tools.staticdir.dir** in *ldapcherry.ini* need to be modified accordingly.
--- a/goodies/demo_backend_configs/attributes.yml
+++ b/goodies/demo_backend_configs/attributes.yml
@ -0,0 +1,112 @@
+description: "First Name and Display Name"
+    display_name: "Display Name"
+    type: string
+    weight: 30
+    autofill:
+        function: lcDisplayName
+        args:
+            - $first-name
+            - $name
+    backends:
+        demo: cn
+first-name:
+    description: "First name of the user"
+    display_name: "First Name"
+    search_displayed: True
+    type: string
+    weight: 20
+    backends:
+        demo: givenName
+name:
+    description: "Family name of the user"
+    display_name: "Name"
+    search_displayed: True
+    weight: 10
+    type: string
+    backends:
+        demo: sn
+email:
+    description: "Email of the user"
+    display_name: "Email"
+    search_displayed: True
+    type: email
+    weight: 40
+    autofill:
+        function: lcMail
+        args:
+            - $first-name
+            - $name
+            - '@example.com'
+    backends:
+        demo: mail
+uid:
+    description: "UID of the user"
+    display_name: "UID"
+    search_displayed: True
+    key: True
+    type: string
+    weight: 50
+    autofill:
+        function: lcUid
+        args:
+            - $first-name
+            - $name
+            - '10000'
+            - '40000'
+    backends:
+        demo: uid
+uidNumber:
+    description: "User ID Number of the user"
+    display_name: "UID Number"
+    weight: 60
+    type: int
+    autofill:
+        function: lcUidNumber
+        args:
+            - $first-name
+            - $name
+            - '10000'
+            - '40000'
+    backends:
+        demo: uidNumber
+gidNumber:
+    description: "Group ID Number of the user"
+    display_name: "GID Number"
+    weight: 70
+    type: int
+    default: '10000'
+    backends:
+        demo: gidNumber
+shell:
+    description: "Shell of the user"
+    display_name: "Shell"
+    weight: 80
+    self: True
+    type: stringlist
+    values:
+        - /bin/bash
+        - /bin/zsh
+        - /bin/sh
+    backends:
+        demo: loginShell
+home:
+    description: "Home user path"
+    display_name: "Home"
+    weight: 90
+    type: string
+    autofill:
+        function: lcHomeDir
+        args:
+            - $first-name
+            - $name
+            - /home/
+    backends:
+        demo: homeDirectory
+password:
+    description: "Password of the user"
+    display_name: "Password"
+    weight: 31
+    self: True
+    type: password
+    backends:
+        demo: userPassword
--- a/goodies/demo_backend_configs/ldapcherry.ini
+++ b/goodies/demo_backend_configs/ldapcherry.ini
@ -0,0 +1,25 @@
+[backends]
+#####################################
+#   configuration of demo backend   #
+#####################################
+
+# Name of the backend
+demo.module = 'ldapcherry.backend.backendDemo'
+# Display name of the Backend
+demo.display_name  = 'Demo Backend'
+# Groups of admin user
+demo.admin.groups  = 'SECOFF'
+# Groups of basic user
+demo.basic.groups  = 'Test 2, Test 1'
+# Password attribute name
+demo.pwd_attr = 'userPassword'
+# Attribute to use for the search
+demo.search_attributes = 'cn, sn, givenName, uid'
+# Login of default admin user
+demo.admin.user = 'admin'
+# Password of default admin user
+demo.admin.password = 'admin'
+# Login of default basic user
+demo.basic.user = 'user'
+# Password of default basic user
+demo.basic.password = 'user'
--- a/goodies/demo_backend_configs/roles.yml
+++ b/goodies/demo_backend_configs/roles.yml
@ -0,0 +1,36 @@
+sec-officer:
+    display_name: Security Officer
+    description: Security officer of the system
+    LC_admins: True
+    backends_groups:
+        demo:
+            - SECOFF
+admin-lv3:
+    display_name: Administrators Level 3
+    description: Super administrators of the system
+    backends_groups:
+        demo:
+            - cn=dns admins,ou=Group,dc=example,dc=org
+            - cn=nagios admins,ou=Group,dc=example,dc=org
+            - cn=puppet admins,ou=Group,dc=example,dc=org
+            - cn=users,ou=Group,dc=example,dc=org
+admin-lv2:
+    display_name: Administrators Level 2
+    description: Basic administrators of the system
+    backends_groups:
+        demo:
+            - cn=nagios admins,ou=Group,dc=example,dc=org
+            - cn=users,ou=Group,dc=example,dc=org
+developpers:
+    display_name: Developpers
+    description: Developpers of the system
+    backends_groups:
+        demo:
+            - cn=developpers,ou=Group,dc=example,dc=org
+            - cn=users,ou=Group,dc=example,dc=org
+users:
+    display_name: Simple Users
+    description: Basic users of the system
+    backends_groups:
+        demo:
+            - cn=users,ou=Group,dc=example,dc=org
--- a/goodies/gen-dev-conf.sh
+++ b/goodies/gen-dev-conf.sh
@ -0,0 +1,18 @@
+#!/bin/sh
+
+ROOT=$(readlink -f $(dirname $0)/../)
+
+cp $ROOT/conf/ldapcherry.ini $ROOT/ldapcherry-dev.ini
+
+sed -i "s|/etc/ldapcherry/|$ROOT/conf/|" $ROOT/ldapcherry-dev.ini
+sed -i "s|/usr/share/ldapcherry/|$ROOT/resources/|" $ROOT/ldapcherry-dev.ini
+sed -i "s|^ldap\.|#ldap.|" $ROOT/ldapcherry-dev.ini
+sed -i "s|#demo\.|ldap.|" $ROOT/ldapcherry-dev.ini
+
+GROUPS='cn=nagios admins\\,ou=Group\\,dc=example\\,dc=org, cn=users\\,ou=Group\\,dc=example\\,dc=org'
+sed -i "s|ldap.admin.groups.*|ldap.admin.groups = '$GROUPS'|" $ROOT/ldapcherry-dev.ini
+
+
+sed -i "s|^min_length.*|min_length = 3|" $ROOT/ldapcherry-dev.ini
+sed -i "s|^min_upper.*|min_upper = 0|" $ROOT/ldapcherry-dev.ini
+sed -i "s|^min_digit.*|min_digit = 0|" $ROOT/ldapcherry-dev.ini
--- a/goodies/nginx-fastcgi.conf
+++ b/goodies/nginx-fastcgi.conf
@ -0,0 +1,25 @@
+server {
+    listen 80 default_server;
+    listen [::]:80 default_server;
+    
+    server_name _;
+    
+    location / {
+        fastcgi_param REQUEST_METHOD $request_method;
+        fastcgi_param QUERY_STRING $query_string;
+        fastcgi_param CONTENT_TYPE $content_type;
+        fastcgi_param CONTENT_LENGTH $content_length;
+        fastcgi_param GATEWAY_INTERFACE CGI/1.1;
+        fastcgi_param SERVER_SOFTWARE nginx/$nginx_version;
+        fastcgi_param REMOTE_ADDR $remote_addr;
+        fastcgi_param REMOTE_PORT $remote_port;
+        fastcgi_param SERVER_ADDR $server_addr;
+        fastcgi_param SERVER_PORT $server_port;
+        fastcgi_param SERVER_NAME $server_name;
+        fastcgi_param SERVER_PROTOCOL $server_protocol;
+        fastcgi_param SCRIPT_FILENAME $fastcgi_script_name;
+        fastcgi_param PATH_INFO $fastcgi_script_name;
+    
+        fastcgi_pass 127.0.0.1:8080;
+    }
+}
--- a/goodies/tag.sh
+++ b/goodies/tag.sh
@ -0,0 +1,8 @@
+#!/bin/sh
+
+cd  `dirname $0`/../
+
+version=`sed -e "s/version\ * = \ *'\(.*\)'.*/\1/;tx;d;:x" ./ldapcherry/version.py`
+
+git tag "$version" -m "version $version"
+git push origin "$version"
--- a/ldapcherry/init.py
+++ b/ldapcherry/init.py
@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 # vim:set expandtab tabstop=4 shiftwidth=4:
 #
@ -8,6 +7,7 @@

 # Generic imports
 import sys
+import os
 import re
 import traceback
 import json
@ -15,10 +15,8 @@ import logging
 import logging.handlers
 from operator import itemgetter
 from socket import error as socket_error
-import base64
-import cgi

-from exceptions import *
+from ldapcherry.exceptions import *
 from ldapcherry.lclogging import *
 from ldapcherry.roles import Roles
 from ldapcherry.attributes import Attributes
@ -31,7 +29,13 @@ from cherrypy.lib.httputil import parse_query_string
 from mako.template import Template
 from mako import lookup
 from mako import exceptions
-from sets import Set
+
+
+if sys.version < '3':
+    from sets import Set as set
+    from urllib import quote_plus
+else:
+    from urllib.parse import quote_plus

 SESSION_KEY = '_cp_username'

@ -56,36 +60,6 @@ class LdapCherry(object):
            traceback=True
            )

-    def _escape_list(self, data):
-        ret = []
-        for i in data:
-            ret.append(cgi.escape(i, True))
-        return ret
-
-    def _escape_dict(self, data):
-        for d in data:
-            if isinstance(data[d], list):
-                data[d] = self._escape_list(data[d])
-            elif isinstance(data[d], dict):
-                data[d] = self._escape_dict(data[d])
-            elif isinstance(data[d], Set):
-                data[d] = Set(self._escape_list(data[d]))
-            else:
-                data[d] = cgi.escape(data[d], True)
-        return data
-
-    def _escape(self, data, dtype):
-        if data is None:
-            return None
-        elif dtype == 'search_list':
-            for d in data:
-                data[d] = self._escape_dict(data[d])
-        elif dtype == 'attr_list':
-            data = self._escape_dict(data)
-        elif dtype == 'lonely_groups':
-            data = self._escape_dict(data)
-        return data
-
    def _get_param(self, section, key, config, default=None):
        """ Get configuration parameter "key" from config
        @str section: the section of the config file
@ -144,10 +118,10 @@ class LdapCherry(object):
        backends = self.backends_params.keys()
        for b in self.roles.get_backends():
            if b not in backends:
-                raise MissingBackend(b)
-        for b in self.roles.get_backends():
+                raise MissingBackend(b, 'role')
+        for b in self.attributes.get_backends():
            if b not in backends:
-                raise MissingBackend(b)
+                raise MissingBackend(b, 'attribute')

    def _init_backends(self, config):
        """ Init all backends
@ -168,7 +142,7 @@ class LdapCherry(object):
            try:
                self.backends_display_names[backend] = \
                    self.backends_params[backend]['display_name']
-            except:
+            except Exception as e:
                self.backends_display_names[backend] = backend
                self.backends_params[backend]['display_name'] = backend
            params = self.backends_params[backend]
@ -178,7 +152,7 @@ class LdapCherry(object):
            except Exception as e:
                raise MissingParameter('backends', backend + '.module')
            try:
-                bc = __import__(module, globals(), locals(), ['Backend'], -1)
+                bc = __import__(module, globals(), locals(), ['Backend'], 0)
            except Exception as e:
                self._handle_exception(e)
                raise BackendModuleLoadingFail(module)
@ -187,7 +161,7 @@ class LdapCherry(object):
                key = self.attributes.get_backend_key(backend)
                self.backends[backend] = bc.Backend(
                    params,
-                    cherrypy.log,
+                    cherrypy.log.error,
                    backend,
                    attrslist,
                    key,
@ -219,8 +193,8 @@ class LdapCherry(object):
            'ldapcherry.ppolicy'
        )
        try:
-            pp = __import__(module, globals(), locals(), ['PPolicy'], -1)
-        except:
+            pp = __import__(module, globals(), locals(), ['PPolicy'], 0)
+        except Exception as e:
            raise BackendModuleLoadingFail(module)
        if 'ppolicy' in config:
            ppcfg = config['ppolicy']
@ -238,7 +212,7 @@ class LdapCherry(object):
        elif self.auth_mode == 'custom':
            # load custom auth module
            auth_module = self._get_param('auth', 'auth.module', config)
-            auth = __import__(auth_module, globals(), locals(), ['Auth'], -1)
+            auth = __import__(auth_module, globals(), locals(), ['Auth'], 0)
            self.auth = auth.Auth(config['auth'], cherrypy.log)
        else:
            raise WrongParamValue(
@ -279,6 +253,15 @@ class LdapCherry(object):
            handler.setFormatter(syslog_formatter)
            cherrypy.log.access_log.addHandler(handler)

+        # if stdout, open a logger on stdout
+        elif access_handler == 'stdout':
+            cherrypy.log.access_log.handlers = []
+            handler = logging.StreamHandler(sys.stdout)
+            formatter = logging.Formatter(
+                'ldapcherry.access - %(levelname)s - %(message)s'
+            )
+            handler.setFormatter(formatter)
+            cherrypy.log.access_log.addHandler(handler)
        # if file, we keep the default
        elif access_handler == 'file':
            pass
@ -324,6 +307,15 @@ class LdapCherry(object):
            handler.setFormatter(syslog_formatter)
            cherrypy.log.error_log.addHandler(handler)

+        # if stdout, open a logger on stdout
+        elif error_handler == 'stdout':
+            cherrypy.log.error_log.handlers = []
+            handler = logging.StreamHandler(sys.stdout)
+            formatter = logging.Formatter(
+                'ldapcherry.app    - %(levelname)s - %(message)s'
+            )
+            handler.setFormatter(formatter)
+            cherrypy.log.error_log.addHandler(handler)
        # if file, we keep the default
        elif error_handler == 'file':
            pass
@ -338,6 +330,7 @@ class LdapCherry(object):
        cherrypy.log.error_log.setLevel(level)

        if debug:
+            cherrypy.log.error_log.handlers = []
            handler = logging.StreamHandler(sys.stderr)
            handler.setLevel(logging.DEBUG)
            cherrypy.log.error_log.addHandler(handler)
@ -387,7 +380,8 @@ class LdapCherry(object):
        )
        # preload templates
        self.temp_lookup = lookup.TemplateLookup(
-            directories=self.template_dir, input_encoding='utf-8'
+            directories=self.template_dir, input_encoding='utf-8',
+            default_filters=['unicode', 'h']
            )
        # load each template
        self.temp = {}
@ -573,7 +567,7 @@ class LdapCherry(object):

    def _check_auth(self, must_admin, redir_login=True):
        """ check if a user is autheticated and, optionnaly an administrator
-        if user not authentifaced -> redirection to login page (with base64
+        if user not authenticated -> redirect to login page (with escaped URL
            of the originaly requested page (redirection after login)
        if user authenticated, not admin and must_admin enabled -> 403 error
        @boolean must_admin: flag "user must be an administrator to access
@ -588,13 +582,13 @@ class LdapCherry(object):
            qs = ''
        else:
            qs = '?' + cherrypy.request.query_string
-        # base64 of the requested URL
-        b64requrl = base64.b64encode(cherrypy.url() + qs)
+        # Escaped version of the requested URL
+        quoted_requrl = quote_plus(cherrypy.url() + qs)
        if not username:
-            # return to login page (with base64 of the url in query string
+            # return to login page (with quoted url in query string)
            if redir_login:
                raise cherrypy.HTTPRedirect(
-                    "/signin?url=%(url)s" % {'url': b64requrl},
+                    "/signin?url=%(url)s" % {'url': quoted_requrl},
                    )
            else:
                raise cherrypy.HTTPError(
@ -606,7 +600,7 @@ class LdapCherry(object):
                or not cherrypy.session['connected']:
            if redir_login:
                raise cherrypy.HTTPRedirect(
-                    "/signin?url=%(url)s" % {'url': b64requrl},
+                    "/signin?url=%(url)s" % {'url': quoted_requrl},
                    )
            else:
                raise cherrypy.HTTPError(
@ -631,7 +625,7 @@ class LdapCherry(object):
        else:
            if redir_login:
                raise cherrypy.HTTPRedirect(
-                    "/signin?url=%(url)s" % {'url': b64requrl},
+                    "/signin?url=%(url)s" % {'url': quoted_requrl},
                    )
            else:
                raise cherrypy.HTTPError(
@ -671,6 +665,7 @@ class LdapCherry(object):
                self._add_notification(
                    'User already exists in backend "' + b + '"'
                    )
+                return
        if not added:
            raise e

@ -694,7 +689,7 @@ class LdapCherry(object):
                roles.append(r)
        groups = self.roles.get_groups(roles)
        for b in groups:
-            self.backends[b].add_to_groups(username, Set(groups[b]))
+            self.backends[b].add_to_groups(username, set(groups[b]))

        cherrypy.log.error(
            msg="user '" + username + "' made member of " +
@ -822,10 +817,10 @@ class LdapCherry(object):
                if b not in g:
                    g[b] = []
            tmp = \
-                Set(groups_add[b]) - \
-                Set(groups_keep[b]) - \
-                Set(groups_current[b]) - \
-                Set(lonely_groups[b])
+                set(groups_add[b]) - \
+                set(groups_keep[b]) - \
+                set(groups_current[b]) - \
+                set(lonely_groups[b])
            cherrypy.log.error(
                msg="user '" + username + "' added to groups: " +
                    str(list(tmp)) + " in backend '" + b + "'",
@ -839,11 +834,11 @@ class LdapCherry(object):
                    g[b] = []
            tmp = \
                (
-                    (Set(groups_rm[b]) | Set(groups_remove[b])) -
-                    (Set(groups_keep[b]) | Set(groups_add[b]))
+                    (set(groups_rm[b]) | set(groups_remove[b])) -
+                    (set(groups_keep[b]) | set(groups_add[b]))
                ) & \
                (
-                    Set(groups_current[b]) | Set(lonely_groups[b])
+                    set(groups_current[b]) | set(lonely_groups[b])
                )
            cherrypy.log.error(
                msg="user '" + username + "' removed from groups: " +
@ -919,7 +914,7 @@ class LdapCherry(object):
            if url is None:
                redirect = "/"
            else:
-                redirect = base64.b64decode(url)
+                redirect = url
            raise cherrypy.HTTPRedirect(redirect)
        else:
            message = "login failed for user '%(user)s'" % {
@ -932,7 +927,7 @@ class LdapCherry(object):
            if url is None:
                qs = ''
            else:
-                qs = '?url=' + url
+                qs = '?url=' + quote_plus(url)
            raise cherrypy.HTTPRedirect("/signin" + qs)

    @cherrypy.expose
@ -969,7 +964,7 @@ class LdapCherry(object):
        return self.temp['index.tmpl'].render(
            is_admin=is_admin,
            attrs_list=attrs_list,
-            searchresult=self._escape(user_attrs, 'attr_list'),
+            searchresult=user_attrs,
            notifications=self._empty_notification(),
            )

@ -985,7 +980,7 @@ class LdapCherry(object):
            res = None
        attrs_list = self.attributes.get_search_attributes()
        return self.temp['searchuser.tmpl'].render(
-            searchresult=self._escape(res, 'search_list'),
+            searchresult=res,
            attrs_list=attrs_list,
            is_admin=is_admin,
            custom_js=self.custom_js,
@ -997,7 +992,7 @@ class LdapCherry(object):
    def checkppolicy(self, **params):
        """ search user page """
        self._check_auth(must_admin=False, redir_login=False)
-        keys = params.keys()
+        keys = list(params.keys())
        if len(keys) != 1:
            cherrypy.response.status = 400
            return "bad argument"
@ -1022,7 +1017,7 @@ class LdapCherry(object):
            res = None
        attrs_list = self.attributes.get_search_attributes()
        return self.temp['searchadmin.tmpl'].render(
-            searchresult=self._escape(res, 'search_list'),
+            searchresult=res,
            attrs_list=attrs_list,
            is_admin=is_admin,
            custom_js=self.custom_js,
@ -1085,7 +1080,7 @@ class LdapCherry(object):
        is_admin = self._check_admin()
        try:
            referer = cherrypy.request.headers['Referer']
-        except:
+        except Exception as e:
            referer = '/'
        self._deleteuser(user)
        self._add_notification('User Deleted')
@ -1104,7 +1099,7 @@ class LdapCherry(object):
            self._add_notification("User modified")
            try:
                referer = cherrypy.request.headers['Referer']
-            except:
+            except Exception as e:
                referer = '/'
            raise cherrypy.HTTPRedirect(referer)

@ -1143,7 +1138,7 @@ class LdapCherry(object):
        try:
            form = self.temp['form.tmpl'].render(
                attributes=self.attributes.attributes,
-                values=self._escape(user_attrs, 'attr_list'),
+                values=user_attrs,
                modify=True,
                keyattr=key,
                autofill=False
@ -1161,10 +1156,7 @@ class LdapCherry(object):
                form=form,
                roles=roles,
                is_admin=is_admin,
-                standalone_groups=self._escape(
-                    standalone_groups,
-                    'lonely_groups'
-                    ),
+                standalone_groups=standalone_groups,
                backends_display_names=self.backends_display_names,
                custom_js=self.custom_js,
                notifications=self._empty_notification(),
@ -1178,7 +1170,7 @@ class LdapCherry(object):

    @cherrypy.expose
    @exception_decorator
-    def default(self, attr='', **params):
+    def default(self, attr='', *args, **params):
        cherrypy.response.status = 404
        self._check_auth(must_admin=False)
        is_admin = self._check_admin()
@ -1219,7 +1211,7 @@ class LdapCherry(object):

            form = self.temp['form.tmpl'].render(
                attributes=self.attributes.get_selfattributes(),
-                values=self._escape(user_attrs, 'attr_list'),
+                values=user_attrs,
                modify=True,
                autofill=False
                )
--- a/ldapcherry/attributes.py
+++ b/ldapcherry/attributes.py
@ -12,9 +12,11 @@ import re
 from ldapcherry.pyyamlwrapper import loadNoDump
 from ldapcherry.pyyamlwrapper import DumplicatedKey
 from ldapcherry.exceptions import *
-from sets import Set
 import yaml

+if sys.version < '3':
+    from sets import Set as set
+
 # List of available types for form
 types = ['string', 'textfield', 'email', 'int', 'stringlist',
         'fix', 'password']
@ -24,14 +26,14 @@ class Attributes:

    def __init__(self, attributes_file):
        self.attributes_file = attributes_file
-        self.backends = Set([])
+        self.backends = set([])
        self.self_attributes = {}
        self.backend_attributes = {}
        self.displayed_attributes = {}
        self.key = None
        try:
            stream = open(attributes_file, 'r')
-        except:
+        except Exception as e:
            raise MissingAttributesFile(attributes_file)
        try:
            self.attributes = loadNoDump(stream)
@ -69,7 +71,7 @@ class Attributes:
            raise MissingUserKey()

    def _is_email(self, email):
-        pattern = '[\.\w]{1,}[@]\w+[.]\w+'
+        pattern = r'[\+\.\w]+@[-\.\w]+\.\w+'
        if re.match(pattern, email):
            return True
        else:
@ -128,7 +130,9 @@ class Attributes:
    def get_backend_attributes(self, backend):
        if backend not in self.backends:
            raise WrongBackend(backend)
-        return self.backend_attributes[backend].keys()
+        ret = list(self.backend_attributes[backend].keys())
+        ret.sort()
+        return ret

    def get_backend_key(self, backend):
        if backend not in self.backends:
--- a/ldapcherry/backend/backendAD.py
+++ b/ldapcherry/backend/backendAD.py
@ -15,6 +15,7 @@ import ldapcherry.backend
 from ldapcherry.exceptions import UserDoesntExist, GroupDoesntExist
 import os
 import re
+import sys


 class CaFileDontExist(Exception):
@ -28,6 +29,7 @@ class MissingAttr(Exception):
        self.log = 'attributes "cn" and "unicodePwd" must be declared ' \
           'in attributes.yml for all Active Directory backends.'

+
 NO_ATTR = 0
 DISPLAYED_ATTRS = 1
 LISTED_ATTRS = 2
@ -128,11 +130,11 @@ class Backend(ldapcherry.backend.backendLdap.Backend):
        self.dn_user_attr = 'cn'
        self.key = 'sAMAccountName'
        self.objectclasses = [
-            'top',
-            'person',
-            'organizationalPerson',
-            'user',
-            'posixAccount',
+            self._byte_p23('top'),
+            self._byte_p23('person'),
+            self._byte_p23('organizationalPerson'),
+            self._byte_p23('user'),
+            self._byte_p23('posixAccount'),
            ]
        self.group_attrs = {
            'member': "%(dn)s"
@ -141,16 +143,25 @@ class Backend(ldapcherry.backend.backendLdap.Backend):
        self.attrlist = []
        self.group_attrs_keys = []
        for a in attrslist:
-            self.attrlist.append(self._str(a))
+            self.attrlist.append(self._byte_p2(a))

-        if 'cn' not in self.attrlist:
+        if self._byte_p2('cn') not in self.attrlist:
            raise MissingAttr()

-        if 'unicodePwd' not in self.attrlist:
+        if self._byte_p2('unicodePwd') not in self.attrlist:
            raise MissingAttr()

+    if sys.version < '3':
+        @staticmethod
+        def _tobyte(in_int):
+            return str(in_int)
+    else:
+        @staticmethod
+        def _tobyte(in_int):
+            return in_int.to_bytes(4, byteorder='big')
+
    def _search_group(self, searchfilter, groupdn):
-        searchfilter = self._str(searchfilter)
+        searchfilter = self._byte_p2(searchfilter)
        ldap_client = self._bind()
        try:
            r = ldap_client.search_s(
@ -182,22 +193,24 @@ class Backend(ldapcherry.backend.backendLdap.Backend):
        ldap_client = self._bind()

        if by_cn:
-            dn = self._str('CN=%(cn)s,%(user_dn)s' % {
+            dn = self._byte_p2('CN=%(cn)s,%(user_dn)s' % {
                        'cn': name,
                        'user_dn': self.userdn
                       })
        else:
-            dn = self._str(name)
+            dn = self._byte_p2(name)

        attrs = {}

-        attrs['unicodePwd'] = self._str(password_value)
+        attrs['unicodePwd'] = self._modlist(self._byte_p2(password_value))

        ldif = modlist.modifyModlist({'unicodePwd': 'tmp'}, attrs)
        ldap_client.modify_s(dn, ldif)

        del(attrs['unicodePwd'])
-        attrs['UserAccountControl'] = str(NORMAL_ACCOUNT)
+        attrs['UserAccountControl'] = self._modlist(
+            self._tobyte(NORMAL_ACCOUNT)
+        )
        ldif = modlist.modifyModlist({'UserAccountControl': 'tmp'}, attrs)
        ldap_client.modify_s(dn, ldif)

@ -211,7 +224,7 @@ class Backend(ldapcherry.backend.backendLdap.Backend):
        if 'unicodePwd' in attrs:
            password = attrs['unicodePwd']
            del(attrs['unicodePwd'])
-            userdn = self._get_user(self._str(username), NO_ATTR)
+            userdn = self._get_user(self._byte_p2(username), NO_ATTR)
            self._set_password(userdn, password, False)
        super(Backend, self).set_attrs(username, attrs)

@ -225,7 +238,7 @@ class Backend(ldapcherry.backend.backendLdap.Backend):

    def get_groups(self, username):
        username = ldap.filter.escape_filter_chars(username)
-        userdn = self._get_user(self._str(username), NO_ATTR)
+        userdn = self._get_user(self._byte_p2(username), NO_ATTR)

        searchfilter = self.group_filter_tmpl % {
            'userdn': userdn,
@ -245,7 +258,7 @@ class Backend(ldapcherry.backend.backendLdap.Backend):
        )

        for entry in groups:
-            ret.append(entry[1]['cn'][0])
+            ret.append(self._uni(entry[1]['cn'][0]))
        return ret

    def auth(self, username, password):
@ -255,8 +268,8 @@ class Backend(ldapcherry.backend.backendLdap.Backend):
            ldap_client = self._connect()
            try:
                ldap_client.simple_bind_s(
-                    self._str(binddn),
-                    self._str(password)
+                    self._byte_p2(binddn),
+                    self._byte_p2(password)
                )
            except ldap.INVALID_CREDENTIALS:
                ldap_client.unbind_s()
--- a/ldapcherry/backend/backendDemo.py
+++ b/ldapcherry/backend/backendDemo.py
@ -7,12 +7,15 @@

 # This is a demo backend

-from sets import Set
+
+import sys
 import ldapcherry.backend
 from ldapcherry.exceptions import UserDoesntExist, \
    GroupDoesntExist, MissingParameter, \
    UserAlreadyExists
 import re
+if sys.version < '3':
+    from sets import Set as set


 class Backend(ldapcherry.backend.Backend):
@ -37,13 +40,17 @@ class Backend(ldapcherry.backend.Backend):
        self.backend_name = name
        admin_user = self.get_param('admin.user', 'admin')
        admin_password = self.get_param('admin.password', 'admin')
-        admin_groups = Set(re.split('\W+', self.get_param('admin.groups')))
+        admin_groups = set(
+            self._basic_splitter(self.get_param('admin.groups'))
+            )
        basic_user = self.get_param('basic.user', 'user')
        basic_password = self.get_param('basic.password', 'user')
-        basic_groups = Set(re.split('\W+', self.get_param('basic.groups')))
+        basic_groups = set(
+            self._basic_splitter(self.get_param('basic.groups'))
+            )
        pwd_attr = self.get_param('pwd_attr')
-        self.search_attrs = Set(
-            re.split('\W+', self.get_param('search_attributes')),
+        self.search_attrs = set(
+            re.split(r'\W+', self.get_param('search_attributes')),
            )
        self.pwd_attr = pwd_attr
        self.admin_user = admin_user
@ -60,6 +67,11 @@ class Backend(ldapcherry.backend.Backend):
                'groups': basic_groups,
                }

+    @staticmethod
+    def _basic_splitter(in_str):
+        return [re.sub(r'(?<!\\)\\', '', x)
+                for x in re.split(r'(?<!\\),\W*', in_str)]
+
    def _check_fix_users(self, username):
        if self.admin_user == username or self.basic_user == username:
            raise Exception('User cannot be modified')
@ -91,7 +103,7 @@ class Backend(ldapcherry.backend.Backend):
        if username in self.users:
            raise UserAlreadyExists(username, self.backend_name)
        self.users[username] = attrs
-        self.users[username]['groups'] = Set([])
+        self.users[username]['groups'] = set([])

    def del_user(self, username):
        """ Delete a user from the backend
@ -103,11 +115,11 @@ class Backend(ldapcherry.backend.Backend):
        self._check_fix_users(username)
        try:
            del self.users[username]
-        except:
+        except Exception as e:
            raise UserDoesntExist(username, self.backend_name)

    def set_attrs(self, username, attrs):
-        """ Set a list of attributes for a given user
+        """ set a list of attributes for a given user

        :param username: 'key' attribute of the user
        :type username: string
@ -128,7 +140,7 @@ class Backend(ldapcherry.backend.Backend):
        """
        self._check_fix_users(username)
        current_groups = self.users[username]['groups']
-        new_groups = current_groups | Set(groups)
+        new_groups = current_groups | set(groups)
        self.users[username]['groups'] = new_groups

    def del_from_groups(self, username, groups):
@ -143,7 +155,7 @@ class Backend(ldapcherry.backend.Backend):
        """
        self._check_fix_users(username)
        current_groups = self.users[username]['groups']
-        new_groups = current_groups - Set(groups)
+        new_groups = current_groups - set(groups)
        self.users[username]['groups'] = new_groups

    def search(self, searchstring):
@ -176,7 +188,7 @@ class Backend(ldapcherry.backend.Backend):
        """
        try:
            return self.users[username]
-        except:
+        except Exception as e:
            raise UserDoesntExist(username, self.backend_name)

    def get_groups(self, username):
@ -188,5 +200,5 @@ class Backend(ldapcherry.backend.Backend):
        """
        try:
            return self.users[username]['groups']
-        except:
+        except Exception as e:
            raise UserDoesntExist(username, self.backend_name)
--- a/ldapcherry/backend/backendLdap.py
+++ b/ldapcherry/backend/backendLdap.py
@ -11,12 +11,16 @@ import ldap.modlist as modlist
 import ldap.filter
 import logging
 import ldapcherry.backend
-from sets import Set
+import sys
 from ldapcherry.exceptions import UserDoesntExist, \
    GroupDoesntExist, \
    UserAlreadyExists
 import os
 import re
+if sys.version < '3':
+    from sets import Set as set
+
+PYTHON_LDAP_MAJOR_VERSION = ldap.__version__[0]


 class CaFileDontExist(Exception):
@ -71,21 +75,21 @@ class Backend(ldapcherry.backend.Backend):
        self.key = key
        # objectclasses parameter is a coma separated list in configuration
        # split it to get a real list, and convert it to bytes
-        for o in re.split('\W+', self.get_param('objectclasses')):
-            self.objectclasses.append(self._str(o))
+        for o in re.split(r'\W+', self.get_param('objectclasses')):
+            self.objectclasses.append(self._byte_p23(o))
        self.group_attrs = {}
-        self.group_attrs_keys = Set([])
+        self.group_attrs_keys = set([])
        for param in config:
            name, sep, group = param.partition('.')
            if name == 'group_attr':
                self.group_attrs[group] = self.get_param(param)
-                self.group_attrs_keys |= Set(
+                self.group_attrs_keys |= set(
                    self._extract_format_keys(self.get_param(param))
                )

        self.attrlist = []
        for a in attrslist:
-            self.attrlist.append(self._str(a))
+            self.attrlist.append(self._byte_p2(a))

    # exception handler (mainly to log something meaningful)
    def _exception_handler(self, e):
@ -128,8 +132,8 @@ class Backend(ldapcherry.backend.Backend):
                    ".groupdn'",
                )
        elif et is ldap.OBJECT_CLASS_VIOLATION:
-            info = e[0]['info']
-            desc = e[0]['desc']
+            info = e.args[0]['info']
+            desc = e.args[0]['desc']
            self._logger(
                severity=logging.ERROR,
                msg="Configuration error, " + desc + ", " + info,
@ -143,7 +147,7 @@ class Backend(ldapcherry.backend.Backend):
                    self.backend_name,
                )
        elif et is ldap.ALREADY_EXISTS:
-            desc = e[0]['desc']
+            desc = e.args[0]['desc']
            self._logger(
                severity=logging.ERROR,
                msg="adding user failed, " + desc,
@ -257,6 +261,16 @@ class Backend(ldapcherry.backend.Backend):
        else:
            attrlist = None

+        self._logger(
+            severity=logging.DEBUG,
+            msg="%(backend)s: executing search "
+                "with filter '%(filter)s' in DN '%(dn)s'" % {
+                    'backend': self.backend_name,
+                    'dn': basedn,
+                    'filter': self._uni(searchfilter)
+                }
+        )
+
        # bind and search the ldap
        ldap_client = self._bind()
        try:
@ -298,7 +312,7 @@ class Backend(ldapcherry.backend.Backend):
        user_filter = self.user_filter_tmpl % {
            'username': self._uni(username)
        }
-        r = self._search(self._str(user_filter), attrs, self.userdn)
+        r = self._search(self._byte_p2(user_filter), attrs, self.userdn)

        if len(r) == 0:
            return None
@ -310,33 +324,88 @@ class Backend(ldapcherry.backend.Backend):
        else:
            dn_entry = r[0]
        return dn_entry
+
    # python-ldap talks in bytes,
    # as the rest of ldapcherry talks in unicode utf-8:
    # * everything passed to python-ldap must be converted to bytes
    # * everything coming from python-ldap must be converted to unicode
-
-    def _str(self, s):
+    #
+    # The previous statement was true for python-ldap < version 3.X.
+    # With versions > 3.0.0 and python 3, it gets tricky,
+    # some parts of python-ldap takes string, specially the filters/escaper.
+    #
+    # so we have now:
+    # *_byte_p2 (unicode -> bytes conversion for python 2)
+    # *_byte_p3 (unicode -> bytes conversion for python 3)
+    # *_byte_p23 (unicode -> bytes conversion for python AND 3)
+    def _byte_p23(self, s):
        """unicode -> bytes conversion"""
        if s is None:
            return None
        return s.encode('utf-8')

-    def _uni(self, s):
-        """bytes -> unicode conversion"""
-        if s is None:
-            return None
-        return s.decode('utf-8', 'ignore')
+    if sys.version < '3':
+        def _byte_p2(self, s):
+            """unicode -> bytes conversion (python 2)"""
+            if s is None:
+                return None
+            return s.encode('utf-8')
+
+        def _byte_p3(self, s):
+            """pass through (does something in python 3)"""
+            return s
+
+        def _uni(self, s):
+            """bytes -> unicode conversion"""
+            if s is None:
+                return None
+            return s.decode('utf-8', 'ignore')
+
+        def attrs_pretreatment(self, attrs):
+            attrs_srt = {}
+            for a in attrs:
+                attrs_srt[self._byte_p2(a)] = self._modlist(
+                    self._byte_p2(attrs[a])
+                )
+            return attrs_srt
+    else:
+        def _byte_p2(self, s):
+            """pass through (does something in python 2)"""
+            return s
+
+        def _byte_p3(self, s):
+            """unicode -> bytes conversion"""
+            if s is None:
+                return None
+            return s.encode('utf-8')
+
+        def _uni(self, s):
+            """bytes -> unicode conversion"""
+            if s is None:
+                return None
+            if type(s) is not str:
+                return s.decode('utf-8', 'ignore')
+            else:
+                return s
+
+        def attrs_pretreatment(self, attrs):
+            attrs_srt = {}
+            for a in attrs:
+                attrs_srt[self._byte_p2(a)] = self._modlist(
+                    self._byte_p3(attrs[a])
+                )
+            return attrs_srt

    def auth(self, username, password):
        """Authentication of a user"""

-        binddn = self._get_user(self._str(username), NO_ATTR)
+        binddn = self._get_user(self._byte_p2(username), NO_ATTR)
        if binddn is not None:
            ldap_client = self._connect()
            try:
                ldap_client.simple_bind_s(
-                        self._str(binddn),
-                        self._str(password)
+                        self._byte_p2(binddn),
+                        self._byte_p2(password)
                        )
            except ldap.INVALID_CREDENTIALS:
                ldap_client.unbind_s()
@ -346,30 +415,35 @@ class Backend(ldapcherry.backend.Backend):
        else:
            return False

-    def attrs_pretreatment(self, attrs):
-        attrs_str = {}
-        for a in attrs:
-            attrs_str[self._str(a)] = self._str(attrs[a])
-        return attrs_str
+    if PYTHON_LDAP_MAJOR_VERSION == '2':
+        @staticmethod
+        def _modlist(in_attr):
+            return in_attr
+
+    else:
+        @staticmethod
+        def _modlist(in_attr):
+            return [in_attr]

    def add_user(self, attrs):
        """add a user"""
        ldap_client = self._bind()
        # encoding crap
-        attrs_str = self.attrs_pretreatment(attrs)
+        attrs_srt = self.attrs_pretreatment(attrs)

-        attrs_str['objectClass'] = self.objectclasses
+        attrs_srt[self._byte_p2('objectClass')] = self.objectclasses
        # construct is DN
        dn = \
-            self._str(self.dn_user_attr) + \
-            '=' + \
-            ldap.dn.escape_dn_chars(
-                self._str(attrs[self.dn_user_attr])
+            self._byte_p2(self.dn_user_attr) + \
+            self._byte_p2('=') + \
+            self._byte_p2(ldap.dn.escape_dn_chars(
+                        attrs[self.dn_user_attr]
+                    )
                ) + \
-            ',' + \
-            self._str(self.userdn)
-        # gen the ldif fir add_s and add the user
-        ldif = modlist.addModlist(attrs_str)
+            self._byte_p2(',') + \
+            self._byte_p2(self.userdn)
+        # gen the ldif first add_s and add the user
+        ldif = modlist.addModlist(attrs_srt)
        try:
            ldap_client.add_s(dn, ldif)
        except ldap.ALREADY_EXISTS as e:
@ -383,7 +457,7 @@ class Backend(ldapcherry.backend.Backend):
        """delete a user"""
        ldap_client = self._bind()
        # recover the user dn
-        dn = self._str(self._get_user(self._str(username), NO_ATTR))
+        dn = self._byte_p2(self._get_user(self._byte_p2(username), NO_ATTR))
        # delete
        if dn is not None:
            ldap_client.delete_s(dn)
@ -393,17 +467,17 @@ class Backend(ldapcherry.backend.Backend):
        ldap_client.unbind_s()

    def set_attrs(self, username, attrs):
-        """ Set user attributes"""
+        """ set user attributes"""
        ldap_client = self._bind()
-        tmp = self._get_user(self._str(username), ALL_ATTRS)
+        tmp = self._get_user(self._byte_p2(username), ALL_ATTRS)
        if tmp is None:
            raise UserDoesntExist(username, self.backend_name)
-        dn = self._str(tmp[0])
+        dn = self._byte_p2(tmp[0])
        old_attrs = tmp[1]
        for attr in attrs:
-            bcontent = self._str(attrs[attr])
-            battr = self._str(attr)
-            new = {battr: bcontent}
+            bcontent = self._byte_p2(attrs[attr])
+            battr = self._byte_p2(attr)
+            new = {battr: self._modlist(self._byte_p3(bcontent))}
            # if attr is dn entry, use rename
            if attr.lower() == self.dn_user_attr.lower():
                ldap_client.rename_s(
@ -420,39 +494,42 @@ class Backend(ldapcherry.backend.Backend):
                    if type(old_attrs[attr]) is list:
                        tmp = []
                        for value in old_attrs[attr]:
-                            tmp.append(self._str(value))
+                            tmp.append(self._byte_p2(value))
                        bold_value = tmp
                    else:
-                        bold_value = self._str(old_attrs[attr])
+                        bold_value = self._modlist(
+                            self._byte_p3(old_attrs[attr])
+                        )
                    old = {battr: bold_value}
                # attribute is not set, just add it
                else:
                    old = {}
                ldif = modlist.modifyModlist(old, new)
-                try:
-                    ldap_client.modify_s(dn, ldif)
-                except Exception as e:
-                    ldap_client.unbind_s()
-                    self._exception_handler(e)
+                if ldif:
+                    try:
+                        ldap_client.modify_s(dn, ldif)
+                    except Exception as e:
+                        ldap_client.unbind_s()
+                        self._exception_handler(e)

        ldap_client.unbind_s()

    def add_to_groups(self, username, groups):
        ldap_client = self._bind()
        # recover dn of the user and his attributes
-        tmp = self._get_user(self._str(username), ALL_ATTRS)
+        tmp = self._get_user(self._byte_p2(username), ALL_ATTRS)
        dn = tmp[0]
        attrs = tmp[1]
        attrs['dn'] = dn
        self._normalize_group_attrs(attrs)
-        dn = self._str(tmp[0])
+        dn = self._byte_p2(tmp[0])
        # add user to all groups
        for group in groups:
-            group = self._str(group)
+            group = self._byte_p2(group)
            # iterate on group membership attributes
            for attr in self.group_attrs:
                # fill the content template
-                content = self._str(self.group_attrs[attr] % attrs)
+                content = self._byte_p2(self.group_attrs[attr] % attrs)
                self._logger(
                    severity=logging.DEBUG,
                    msg="%(backend)s: adding user '%(user)s'"
@ -466,11 +543,14 @@ class Backend(ldapcherry.backend.Backend):
                            'backend': self.backend_name
                            }
                )
-                ldif = modlist.modifyModlist({}, {attr: content})
+                ldif = modlist.modifyModlist(
+                        {},
+                        {attr: self._modlist(self._byte_p3(content))}
+                       )
                try:
                    ldap_client.modify_s(group, ldif)
                # if already member, not a big deal, just log it and continue
-                except ldap.TYPE_OR_VALUE_EXISTS as e:
+                except (ldap.TYPE_OR_VALUE_EXISTS, ldap.ALREADY_EXISTS) as e:
                    self._logger(
                        severity=logging.INFO,
                        msg="%(backend)s: user '%(user)s'"
@ -494,19 +574,19 @@ class Backend(ldapcherry.backend.Backend):
        # it follows the same logic than add_to_groups
        # but with MOD_DELETE
        ldap_client = self._bind()
-        tmp = self._get_user(self._str(username), ALL_ATTRS)
+        tmp = self._get_user(self._byte_p2(username), ALL_ATTRS)
        if tmp is None:
            raise UserDoesntExist(username, self.backend_name)
        dn = tmp[0]
        attrs = tmp[1]
        attrs['dn'] = dn
        self._normalize_group_attrs(attrs)
-        dn = self._str(tmp[0])
+        dn = self._byte_p2(tmp[0])
        for group in groups:
-            group = self._str(group)
+            group = self._byte_p2(group)
            for attr in self.group_attrs:
-                content = self._str(self.group_attrs[attr] % attrs)
-                ldif = [(ldap.MOD_DELETE, attr, content)]
+                content = self._byte_p2(self.group_attrs[attr] % attrs)
+                ldif = [(ldap.MOD_DELETE, attr, self._byte_p3(content))]
                try:
                    ldap_client.modify_s(group, ldif)
                except ldap.NO_SUCH_ATTRIBUTE as e:
@ -529,7 +609,9 @@ class Backend(ldapcherry.backend.Backend):
    def search(self, searchstring):
        """Search users"""
        # escape special char to avoid injection
-        searchstring = ldap.filter.escape_filter_chars(self._str(searchstring))
+        searchstring = ldap.filter.escape_filter_chars(
+            self._byte_p2(searchstring)
+        )
        # fill the search string template
        searchfilter = self.search_filter_tmpl % {
            'searchstring': searchstring
@ -554,7 +636,7 @@ class Backend(ldapcherry.backend.Backend):
    def get_user(self, username):
        """Gest a specific user"""
        ret = {}
-        tmp = self._get_user(self._str(username), ALL_ATTRS)
+        tmp = self._get_user(self._byte_p2(username), ALL_ATTRS)
        if tmp is None:
            raise UserDoesntExist(username, self.backend_name)
        attrs_tmp = tmp[1]
@ -568,7 +650,7 @@ class Backend(ldapcherry.backend.Backend):

    def get_groups(self, username):
        """Get all groups of a user"""
-        username = ldap.filter.escape_filter_chars(self._str(username))
+        username = ldap.filter.escape_filter_chars(self._byte_p2(username))
        userdn = self._get_user(username, NO_ATTR)

        searchfilter = self.group_filter_tmpl % {
@ -579,5 +661,5 @@ class Backend(ldapcherry.backend.Backend):
        groups = self._search(searchfilter, NO_ATTR, self.groupdn)
        ret = []
        for entry in groups:
-            ret.append(entry[0])
+            ret.append(self._uni(entry[0]))
        return ret
--- a/scripts/ldapcherryd
+++ b/scripts/ldapcherryd
@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # vim:set expandtab tabstop=4 shiftwidth=4:
 # The MIT License (MIT)
@ -95,14 +95,14 @@ def start(configfile=None, daemonize=False, environment=None,
    # Always start the engine; this will start all other services
    try:
        engine.start()
-    except:
+    except Exception as e:
        # Assume the error has been logged already via bus.log.
        sys.exit(1)
    else:
        engine.block()


-if __name__ == '__main__':
+def main():
    from optparse import OptionParser

    p = OptionParser()
@ -142,3 +142,7 @@ if __name__ == '__main__':
    start(options.config, options.daemonize,
          options.environment, options.fastcgi, options.scgi,
          options.pidfile, options.cgi, options.debug)
+
+
+if __name__ == '__main__':
+    main()
--- a/ldapcherry/exceptions.py
+++ b/ldapcherry/exceptions.py
@ -46,11 +46,12 @@ class MissingRole(Exception):


 class MissingBackend(Exception):
-    def __init__(self, backend):
+    def __init__(self, backend, type_conf):
        self.backend = backend
        self.log = \
-            "backend '%(backend)s' does not exist in main config file" % \
-            {'backend': backend}
+            "backend '%(backend)s' does not exist in main config file " \
+            "but is still declared in '%(type_conf)s' file" % \
+            {'backend': backend, 'type_conf': type_conf}


 class WrongBackend(Exception):
--- a/ldapcherry/lclogging.py
+++ b/ldapcherry/lclogging.py
@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 # vim:set expandtab tabstop=4 shiftwidth=4:
 #
--- a/ldapcherry/roles.py
+++ b/ldapcherry/roles.py
@ -9,12 +9,14 @@ import os
 import sys
 import copy

-from sets import Set
 from ldapcherry.pyyamlwrapper import loadNoDump
 from ldapcherry.pyyamlwrapper import DumplicatedKey
 from ldapcherry.exceptions import *
 import yaml

+if sys.version < '3':
+    from sets import Set as set
+

 class CustomDumper(yaml.SafeDumper):
    "A custom YAML dumper that never emits aliases"
@ -27,10 +29,10 @@ class Roles:

    def __init__(self, role_file):
        self.role_file = role_file
-        self.backends = Set([])
+        self.backends = set([])
        try:
            stream = open(role_file, 'r')
-        except:
+        except Exception as e:
            raise MissingRolesFile(role_file)
        try:
            self.roles_raw = loadNoDump(stream)
@ -51,11 +53,12 @@ class Roles:
        for backends in backends_list:
            for b in backends:
                if b not in ret:
-                    ret[b] = Set([])
+                    ret[b] = set([])
                for group in backends[b]:
                    ret[b].add(group)
        for b in ret:
            ret[b] = list(ret[b])
+            ret[b].sort()
        return ret

    def _flatten(self, roles=None, groups=None):
@ -134,8 +137,8 @@ class Roles:

            if roleid not in self.graph:
                self.graph[roleid] = {
-                    'parent_roles': Set([]),
-                    'sub_roles': Set([])
+                    'parent_roles': set([]),
+                    'sub_roles': set([])
                    }

        # Create the nested groups
@ -147,7 +150,7 @@ class Roles:
                    if b not in self.group2roles:
                        self.group2roles[b] = {}
                    if g not in self.group2roles[b]:
-                        self.group2roles[b][g] = Set([])
+                        self.group2roles[b][g] = set([])
                    self.group2roles[b][g].add(roleid)

            parent_roles[roleid] = []
@ -223,7 +226,7 @@ class Roles:
        # add groups of the role to usedgroups
        for b in self.roles[role]['backends_groups']:
            if b not in usedgroups:
-                usedgroups[b] = Set([])
+                usedgroups[b] = set([])
            for g in self.roles[role]['backends_groups'][b]:
                usedgroups[b].add(g)

@ -254,11 +257,11 @@ class Roles:
        """get groups to remove from list of
        roles to remove and current roles
        """
-        current_roles = Set(current_roles)
+        current_roles = set(current_roles)

        ret = {}
-        roles_to_remove = Set(roles_to_remove)
-        tmp = Set([])
+        roles_to_remove = set(roles_to_remove)
+        tmp = set([])
        # get sub roles of the role to remove that the user belongs to
        # if we remove a role, there is no reason to keep the sub roles
        for r in roles_to_remove:
@ -267,7 +270,7 @@ class Roles:
                    tmp.add(sr)

        roles_to_remove = roles_to_remove.union(tmp)
-        roles = current_roles.difference(Set(roles_to_remove))
+        roles = current_roles.difference(set(roles_to_remove))
        groups_roles = self._get_groups(roles)
        groups_roles_to_remove = self._get_groups(roles_to_remove)

@ -284,12 +287,12 @@ class Roles:
            for b in self.flatten[r]['backends_groups']:
                groups = self.flatten[r]['backends_groups'][b]
                if b not in ret:
-                    ret[b] = Set(groups)
-                ret[b] = ret[b].union(Set(groups))
+                    ret[b] = set(groups)
+                ret[b] = ret[b].union(set(groups))
        return ret

    def _get_subroles(self, role):
-        ret = Set([])
+        ret = set([])
        for sr in self.graph[role]['sub_roles']:
            tmp = self._get_subroles(sr)
            tmp.add(sr)
@ -298,10 +301,10 @@ class Roles:

    def get_roles(self, groups):
        """get list of roles and list of standalone groups"""
-        roles = Set([])
-        parentroles = Set([])
-        notroles = Set([])
-        tmp = Set([])
+        roles = set([])
+        parentroles = set([])
+        notroles = set([])
+        tmp = set([])
        usedgroups = {}
        unusedgroups = {}
        ret = {}
@ -316,7 +319,7 @@ class Roles:
            for g in groups[b]:
                if b not in usedgroups or g not in usedgroups[b]:
                    if b not in unusedgroups:
-                        unusedgroups[b] = Set([])
+                        unusedgroups[b] = set([])
                    unusedgroups[b].add(g)

        ret['roles'] = roles
--- a/ldapcherry/version.py
+++ b/ldapcherry/version.py
@ -0,0 +1,8 @@
+# -*- coding: utf-8 -*-
+# vim:set expandtab tabstop=4 shiftwidth=4:
+#
+# The MIT License (MIT)
+# ldapCherry
+# Copyright (c) 2014 Carpentier Pierre-Francois
+
+version = '1.1.1'
--- a/misc/debug_ldapbackend.py
+++ b/misc/debug_ldapbackend.py
@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-

 from __future__ import with_statement
--- a/misc/debug_roles.py
+++ b/misc/debug_roles.py
@ -30,7 +30,7 @@ groups = {
 'ad' : ['Domain Users', 'Domain Users 2'],
 'ldap': ['cn=users,ou=group,dc=example,dc=com',
    'cn=nagios admins,ou=group,dc=example,dc=com',
-    'cn=developpers,ou=group,dc=example,dc=com',
+    'cn=developers,ou=group,dc=example,dc=com',
    ],
 'toto': ['not a group'],
 }
--- a/requirements-el7.txt
+++ b/requirements-el7.txt
@ -0,0 +1,4 @@
+CherryPy>=3.0.0
+PyYAML
+Mako
+python-ldap==3.4.0
--- a/requirements-stretch.txt
+++ b/requirements-stretch.txt
@ -0,0 +1,4 @@
+CherryPy>=3.0.0
+PyYAML
+Mako
+python-ldap==3.4.0
--- a/resources/static/js/alignforms.js
+++ b/resources/static/js/alignforms.js
@ -1,21 +1,23 @@
-classes = ['lcform-col-1', 'lcform-col-2']; 
-for(var j in classes){
+var classes = ["lcform-col-1", "lcform-col-2"];
+var j, lenj = classes.length;
+for(j=0; j < lenj; j++){
    var formSelector = classes[j];
    var forms = document.getElementsByClassName(formSelector);
    //console.log(formSelector);
    //console.log(forms);
    if (forms.length > 0){
-        var in_groups = forms[0].getElementsByClassName('input-group-addon');
-        //console.log(in_groups);
-        var i, len = in_groups.length;
+        forms[0].style.removeProperty("display");
+        var InputGroups = forms[0].getElementsByClassName("input-group-addon");
+        //console.log(InputGroups);
+        var i, len = InputGroups.length;
        var longest = 0;
        for(i=0; i < len; i++){
-            if (in_groups[i].id != 'basic-addon-password2'){
-                longest = longest < in_groups[i].clientWidth ? in_groups[i].clientWidth : longest;
+            if (InputGroups[i].id !== "basic-addon-password2"){
+                longest = longest < InputGroups[i].clientWidth ? InputGroups[i].clientWidth : longest;
            }
        }
        for(i=0; i < len; i++){
-            in_groups[i].style.minWidth = (longest + 0) + 'px';
+            InputGroups[i].style.minWidth = (longest + 0) + "px";
        }
        //console.log(longest);
    }
--- a/resources/static/js/lc-filler.js
+++ b/resources/static/js/lc-filler.js
@ -30,3 +30,8 @@ function lcUidNumber(firstname, lastname, minuid, maxuid){
 function lcHomeDir(firstname, lastname, basedir){
    return basedir+lcUid(firstname, lastname);
 }
+
+function lcCopy(value){
+    return value;
+}
+
--- a/resources/templates/adduser.tmpl
+++ b/resources/templates/adduser.tmpl
@ -6,14 +6,14 @@
        </div>
        <div class="col-md-12 column">
            <div class="well well-sm">
-              <form method='POST' autocomplete="off" action='/adduser' role="form" class="form-signin" id=form>
+              <form method='POST' autocomplete="off" action='/adduser' role="form" class="form-signin" id="form">
              <fieldset>
              <legend>Fill new user's attributes:</legend>
-              ${form}
+              ${form | n}
              </fieldset>
              <fieldset>
              <legend>Enable/Disable user's roles:</legend>
-              ${roles}
+              ${roles | n}
              </fieldset>
              <div class="form-group">
                <div class="input-group">
--- a/resources/templates/base.tmpl
+++ b/resources/templates/base.tmpl
@ -58,7 +58,7 @@
 <body>
 % if notifications:
 % for notif in notifications:
-<script type="text/javascript">$.notify('${notif}')</script>
+<script type="text/javascript">$.notify('${notif | n}')</script>
 % endfor
 % endif
 <div class="container">
@ -70,6 +70,6 @@
        <p class="muted credit"><a href="http://ldapcherry.readthedocs.org" target="_blank">LdapCherry</a> • © 2016 • Pierre-François Carpentier • Released under the MIT License</p>
      </div>
    </div>
-</body>
 <script type="text/javascript" src="/static/js/alignforms.js"></script>
+</body>
 </html>
--- a/resources/templates/form.tmpl
+++ b/resources/templates/form.tmpl
@ -1,5 +1,6 @@
 ## -*- coding: utf-8 -*-
 <% 
+from markupsafe import Markup
 len_attr = len(attributes)
 switch = len_attr / 2
 if not switch * 2 == len_attr:
@ -31,32 +32,32 @@ for a in sorted(attributes.keys(), key=lambda attr: attributes[attr]['weight']):
            raw_value = values[a]
        if raw_value is None:
           raw_value = ''
-        value = ' value="'+ raw_value + '"'
-        value2 = '<option>'+ raw_value +'</option>'
+        value = Markup(' value="{}"').format(raw_value)
+        value2 = Markup('<option>{}</option>').format(raw_value)
    else:
        raw_value = ''
        value = ''
        value2 = ''
    if 'default' in attr and value == '':
-        value = ' value="'+ attr['default'] + '"'
+        value = Markup(' value="{}"').format(attr['default'])
  %>

  <span class="input-group-addon" id="basic-addon-${a}">${attr['display_name']}</span>
    % if modify and a == keyattr:
-  <input type="hidden" id="attr.${a}" name="attr.${a}" class="form-control" autocomplete='off' aria-describedby="basic-addon-${a}" ${required} ${value} readonly  onfocus="this.removeAttribute('readonly');">
+  <input type="hidden" id="attr.${a}" name="attr.${a}" class="form-control" autocomplete='off' aria-describedby="basic-addon-${a}" ${required} ${value | n} readonly  onfocus="this.removeAttribute('readonly');">
  <span class="form-control" aria-describedby="basic-addon-${a}">${raw_value}</span>
    % elif attr['type'] == 'string':
-  <input type="text"   id="attr.${a}" name="attr.${a}" class="form-control" autocomplete='off' placeholder="${attr['description']}" aria-describedby="basic-addon-${a}" ${required} ${value} readonly  onfocus="this.removeAttribute('readonly');">
+  <input type="text"   id="attr.${a}" name="attr.${a}" class="form-control" autocomplete='off' placeholder="${attr['description']}" aria-describedby="basic-addon-${a}" ${required} ${value | n} readonly  onfocus="this.removeAttribute('readonly');">
    % elif attr['type'] == 'email':
-  <input type="email"  id="attr.${a}" name="attr.${a}" class="form-control" autocomplete='off' placeholder="${attr['description']}" aria-describedby="basic-addon-${a}" ${required} ${value} data-error="email address is invalid" readonly  onfocus="this.removeAttribute('readonly');">
+  <input type="email"  id="attr.${a}" name="attr.${a}" class="form-control" autocomplete='off' placeholder="${attr['description']}" aria-describedby="basic-addon-${a}" ${required} ${value | n} data-error="email address is invalid" readonly  onfocus="this.removeAttribute('readonly');">
    % elif attr['type'] == 'int':
-  <input type="number" id="attr.${a}" name="attr.${a}" class="form-control" autocomplete='off' placeholder="${attr['description']}" aria-describedby="basic-addon-${a}" ${required} ${value} readonly  onfocus="this.removeAttribute('readonly');">
+  <input type="number" id="attr.${a}" name="attr.${a}" class="form-control" autocomplete='off' placeholder="${attr['description']}" aria-describedby="basic-addon-${a}" ${required} ${value | n} readonly  onfocus="this.removeAttribute('readonly');">
    % elif attr['type'] == 'fix':
  <input type="hidden" id="attr.${a}" name="attr.${a}" class="form-control" autocomplete='off' aria-describedby="basic-addon-${a}" ${required} value="${attr['value']}" readonly  onfocus="this.removeAttribute('readonly');">
  <span class="form-control" placeholder="${attr['description']}" aria-describedby="basic-addon-${a}">${attr['value']}</span>
    % elif attr['type'] == 'stringlist':
  <select class="form-control" id="attr.${a}" name="attr.${a}">
-        ${value2}
+        ${value2 | n}
        %for val in attr['values']:
        %if '<option>' + val + '</option>' != value2:
        <option>${val}</option>
@ -76,39 +77,40 @@ for a in sorted(attributes.keys(), key=lambda attr: attributes[attr]['weight']):
 % endfor
 </%def>
 <div class="row">
-<div class="col-md-6 column lcform-col-1">
+<div class="col-md-6 column lcform-col-1" style="display:none;">
 ${form_col(lc1)}
 </div>
-<div class="col-md-6 column lcform-col-2">
+<div class="col-md-6 column lcform-col-2" style="display:none;">
 ${form_col(lc2)}
 </div>
 </div>
 % if autofill:
 <%
-from sets import Set
-attr_set = Set([])
+attr_set = []
 attr_events = {}
 functions = {}
 for attrid in attributes:
    attr = attributes[attrid]
    field = 'attr.' + attrid
-    attr_set.add(field)
-    if 'autofill' in attr:
-        function = attr['autofill']['function']
-        tuple = (field, function)
-        if not tuple in functions:
-            functions[tuple] = []
-        for arg in attr['autofill']['args']:
-            if arg[0] == '$':
-                field_arg = 'attr.' + arg[1:]
-                attr_set.add(field_arg)
-                functions[tuple].append("fields['" + field_arg + "'].value")
-                if not field_arg in attr_events:
-                    attr_events[field_arg] = []
-                attr_events[field_arg].append(tuple) 
-            else:
-                value = arg
-                functions[tuple].append("'" + value + "'")
+    if field not in attr_set:
+        attr_set.append(field)
+        if 'autofill' in attr:
+            function = attr['autofill']['function']
+            tuple = (field, function)
+            if not tuple in functions:
+                functions[tuple] = []
+            for arg in attr['autofill']['args']:
+                if arg[0] == '$':
+                    field_arg = 'attr.' + arg[1:]
+                    if field_arg not in attr_set:
+                        attr_set.append(field_arg)
+                    functions[tuple].append("fields['" + field_arg + "'].value")
+                    if not field_arg in attr_events:
+                        attr_events[field_arg] = []
+                    attr_events[field_arg].append(tuple)
+                else:
+                    value = arg
+                    functions[tuple].append("'" + value + "'")
 %>
 <script>
 var fields = new Object();
@ -121,7 +123,7 @@ if (fields['${attrid}'] != null) {
    fields['${attrid}'].onchange = function () {
    % for tuple in attr_events[attrid]:
        if (typeof(${tuple[1]}) == "function") {
-            fields['${tuple[0]}'].value = ${tuple[1]}(${', '.join(functions[tuple])});
+            fields['${tuple[0]}'].value = ${tuple[1] | n}(${', '.join(functions[tuple]) | n});
        }
    % endfor
    };
--- a/resources/templates/index.tmpl
+++ b/resources/templates/index.tmpl
@ -11,8 +11,8 @@
              </div>
              <div class="panel-body">
 		<table id="RecordTable" class="table table-hover table-condensed">
-		<tbody>
 		% if not searchresult is None:
+		<tbody>
                     %for attr in sorted(attrs_list.keys(), key=lambda attr: attrs_list[attr]['weight']):
                     <tr>
                             % if attr in searchresult:
@ -26,8 +26,8 @@
                             % endif
                     </tr>
                     % endfor
-		%endif
 		</tbody>
+		%endif
                </table>
                </div>
              </div>
--- a/resources/templates/login.tmpl
+++ b/resources/templates/login.tmpl
@ -4,13 +4,13 @@
    <div class="row clearfix" style="margin-top:30px">
        <div class="col-md-4 column"></div>
        <div class="col-md-4 column well">
-<%
-if url is None:
-    qs=''
-else:
-    qs='?url=' + url
-%>
-            <form method='POST' action='/login${qs}' role="form" class="form-signin">
+            <form method='POST' role="form" class="form-signin"
+% if url:
+                action='login?url=${url | u}'
+% else:
+                action='login'
+% endif
+            >
              <div class="form-group">
              <h2 class="form-signin-heading">Please sign in</h2>
                <div class="input-group">
--- a/resources/templates/modify.tmpl
+++ b/resources/templates/modify.tmpl
@ -9,11 +9,11 @@
              <form method='POST' action='/modify' role="form" class="form-signin" id="form">
              <fieldset>
              <legend>Modify user's attributes:</legend>
-              ${form}
+              ${form | n}
              </fieldset>
              <fieldset>
              <legend>Enable/Disable user's roles:</legend>
-              ${roles}
+              ${roles | n}
              </fieldset>
              % if len(standalone_groups) != 0:
              <fieldset>
--- a/resources/templates/roles.tmpl
+++ b/resources/templates/roles.tmpl
@ -1,7 +1,7 @@
 ## -*- coding: utf-8 -*-
 <script type="text/javascript">
-    var graph = ${graph_js};
-    var roles = ${roles_js};
+    var graph = ${graph_js | n};
+    var roles = ${roles_js | n};
    function enableParentRoles(roleid){
        var parentRoles = graph[roleid]['parent_roles'];
        var DnRole = roles[roleid];
--- a/resources/templates/searchadmin.tmpl
+++ b/resources/templates/searchadmin.tmpl
@ -52,10 +52,10 @@
                                </td>
                            % endfor
                                <td>
-                                    <a href="/modify?user=${user}" class="btn btn-xs blue pad" ><span class="glyphicon glyphicon-cog"></span> Modify</a>
+                                    <a href="/modify?user=${user | n,u}" class="btn btn-xs blue pad" ><span class="glyphicon glyphicon-cog"></span> Modify</a>
                                </td>
                                <td>
-                                    <a href="/delete?user=${user}" data-toggle='confirmation-delete' class="btn btn-xs red pad"><span class="glyphicon glyphicon-remove-sign"></span> Delete</a>
+                                    <a href="/delete?user=${user | n,u}" data-toggle='confirmation-delete' class="btn btn-xs red pad"><span class="glyphicon glyphicon-remove-sign"></span> Delete</a>
                                </td>
                            </tr>
                            % endfor
--- a/resources/templates/selfmodify.tmpl
+++ b/resources/templates/selfmodify.tmpl
@ -8,7 +8,7 @@
            <div class="well well-sm">
              <form method='POST' action='/selfmodify' autocomplete="off" role="form" class="form-signin" id="form">
              <legend>Modify your attributes:</legend>
-              ${form}
+              ${form | n}
              </fieldset>
              <div class="form-group">
                <div class="input-group">
--- a/run_test.sh
+++ b/run_test.sh
@ -5,11 +5,11 @@ Gre='\33[0;32m';
 RCol='\33[0m';

 cd `dirname $0`
-python setup.py test &&\
-printf "\nPEP 8 compliance check:\n\n"
-pep8 \
-    --repeat \
-    --show-source \
-    --exclude=.venv,.tox,dist,docs,build,*.egg,tests,misc . && \
-    printf "[${Gre}Passed${RCol}] Yeah! everything is clean\n\n" || \
-    printf "[${Red}Failed${RCol}] Oh No! there is some mess to fix\n\n"
+python3 setup.py test #&&\
+#printf "\nPEP 8 compliance check:\n\n"
+#pep8 \
+#    --recurssive ./ \
+#    --show-source \
+#    --exclude=.venv,.tox,dist,docs,build,*.egg,tests,misc . && \
+#    printf "[${Gre}Passed${RCol}] Yeah! everything is clean\n\n" || \
+#    printf "[${Red}Failed${RCol}] Oh No! there is some mess to fix\n\n"
--- a/setup.py
+++ b/setup.py
@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 # vim:set expandtab tabstop=4 shiftwidth=4:

@ -17,17 +17,24 @@ data_dir = os.path.join(datarootdir, 'ldapcherry')
 config_dir = os.path.join(sysconfdir, 'ldapcherry')
 small_description = 'A simple web application to manage Ldap entries'

+sys.path.append('ldapcherry/')
+from version import version
+
 # change requirements according to python version
 if sys.version_info[0] == 2:
+    install_requires = [
+        'CherryPy >= 3.0.0,< 18.0.0',
+        'python-ldap',
+        'PyYAML',
+        'Mako'
+        ],
+elif sys.version_info[0] == 3:
    install_requires = [
        'CherryPy >= 3.0.0',
        'python-ldap',
        'PyYAML',
        'Mako'
        ],
-elif sys.version_info[0] == 3:
-    print('unsupported version')
-    exit(1)
 else:
    print('unsupported version')
    exit(1)
@ -108,7 +115,7 @@ if as_option_root() or not os.path.exists(
 setup(
    name='ldapcherry',
    zip_safe=False,
-    version='0.5.0',
+    version=version,
    author='Pierre-Francois Carpentier',
    author_email='carpentier.pf@gmail.com',
    packages=[
@ -117,16 +124,18 @@ setup(
        'ldapcherry.ppolicy'
        ],
    data_files=resources_files,
-    scripts=['scripts/ldapcherryd'],
+    entry_points = {
+        'console_scripts': ['ldapcherryd = ldapcherry.cli:main']
+    },
    url='https://github.com/kakwa/ldapcherry',
    license=license,
    description=small_description,
    long_description=description,
    install_requires=install_requires,
-    tests_require=['pytest', 'pep8'],
+    tests_require=['pytest', 'pep8', 'pytidylib'],
    cmdclass={'test': PyTest},
    classifiers=[
-        'Development Status :: 3 - Alpha',
+        'Development Status :: 5 - Production/Stable',
        'Environment :: Web Environment',
        'Framework :: CherryPy',
        'Intended Audience :: System Administrators',
--- a/tests/cfg/nested.yml
+++ b/tests/cfg/nested.yml
@ -21,10 +21,10 @@ users:
          display_name: Administrators Level 3
          description: description
          subroles: {}
-    developpers:
+    developers:
      backends_groups:
        ad: [Domain Users]
-        ldap: ['cn=developpers,ou=group,dc=example,dc=com']
+        ldap: ['cn=developers,ou=group,dc=example,dc=com']
      display_name: Developpers
      description: description
      subroles: {}
--- a/tests/cfg/roles.yml
+++ b/tests/cfg/roles.yml
@ -23,12 +23,12 @@ admin-lv2:
        ad:
            - Domain Users

-developpers:
+developers:
    display_name: Developpers
    description: description
    backends_groups:
        ldap:
-            - cn=developpers,ou=group,dc=example,dc=com
+            - cn=developers,ou=group,dc=example,dc=com
            - cn=users,ou=group,dc=example,dc=com
        ad:
            - Domain Users
--- a/tests/cfg/roles_adldap.yml
+++ b/tests/cfg/roles_adldap.yml
@ -23,12 +23,12 @@ admin-lv2:
        ad:
            - Administrators

-#developpers:
+#developers:
 #    display_name: Developpers
 #    description: Developpers of the system
 #    backends_groups:
 #        ldap:
-#            - cn=developpers,ou=Group,dc=example,dc=org
+#            - cn=developers,ou=Group,dc=example,dc=org
 #            - cn=users,ou=Group,dc=example,dc=org

 #users:
--- a/tests/cfg/roles_content_dup.yml
+++ b/tests/cfg/roles_content_dup.yml
@ -23,12 +23,12 @@ admin    -lv2:
        ad:
            - Domain Users

-developpers:
+developers:
    display_name: Developpers
    description: description
    backends_groups:
        ldap:
-            - cn=developpers,ou=group,dc=example,dc=com
+            - cn=developers,ou=group,dc=example,dc=com
            - cn=users,ou=group,dc=example,dc=com
        ad:
            - Domain Users
--- a/tests/cfg/roles_key_dup.yml
+++ b/tests/cfg/roles_key_dup.yml
@ -23,12 +23,12 @@ admin    -lv3:
        ad:
            - Domain Users

-developpers:
+developers:
    display_name: Developpers
    description: description
    backends_groups:
        ldap:
-            - cn=developpers,ou=group,dc=example,dc=com
+            - cn=developers,ou=group,dc=example,dc=com
            - cn=users,ou=group,dc=example,dc=com
        ad:
            - Domain Users
--- a/tests/cfg/roles_missing_backends.yml
+++ b/tests/cfg/roles_missing_backends.yml
@ -17,12 +17,12 @@ admin-lv2:
    display_name: Administrators Level 2
    description: description

-developpers:
+developers:
    display_name: Developpers
    description: description
    backends_groups:
        ldap:
-            - cn=developpers,ou=group,dc=example,dc=com
+            - cn=developers,ou=group,dc=example,dc=com
            - cn=users,ou=group,dc=example,dc=com
        ad:
            - Domain Users
--- a/tests/cfg/roles_missing_diplay_name.yml
+++ b/tests/cfg/roles_missing_diplay_name.yml
@ -22,12 +22,12 @@ admin-lv2:
        ad:
            - Domain Users

-developpers:
+developers:
    display_name: Developpers
    description: description
    backends_groups:
        ldap:
-            - cn=developpers,ou=group,dc=example,dc=com
+            - cn=developers,ou=group,dc=example,dc=com
            - cn=users,ou=group,dc=example,dc=com
        ad:
            - Domain Users
--- a/tests/cfg/roles_test.yml
+++ b/tests/cfg/roles_test.yml
@ -17,12 +17,12 @@ admin-lv2:
            - cn=nagios admins,ou=Group,dc=example,dc=org
            - cn=users,ou=Group,dc=example,dc=org

-developpers:
+developers:
    display_name: Developpers
    description: Developpers of the system
    backends_groups:
        ldap:
-            - cn=developpers,ou=Group,dc=example,dc=org
+            - cn=developers,ou=Group,dc=example,dc=org
            - cn=users,ou=Group,dc=example,dc=org

 users:
--- a/tests/disable.py
+++ b/tests/disable.py
@ -1,7 +1,7 @@
 import os
 def travis_disabled(f):
    def _decorator(f):
-            print 'test has been disabled on travis'
+            print('test has been disabled on travis')
    if 'TRAVIS' in os.environ and os.environ['TRAVIS'] == 'yes':
        return _decorator
    else:
@ -9,7 +9,7 @@ def travis_disabled(f):

 def slow_disabled(f):
    def _decorator(f):
-            print 'test has been disabled by env var LCNOSLOW'
+            print('test has been disabled by env var LCNOSLOW')
    if 'LCNOSLOW' in os.environ and os.environ['LCNOSLOW'] == 'yes':
        return _decorator
    else:
--- a/tests/html_validator.py
+++ b/tests/html_validator.py
@ -1,243 +0,0 @@
-#!/usr/bin/python
-
-# Copyright (c) 2007-2008 Mozilla Foundation
-#
-# Permission is hereby granted, free of charge, to any person obtaining a
-# copy of this software and associated documentation files (the "Software"),
-# to deal in the Software without restriction, including without limitation
-# the rights to use, copy, modify, merge, publish, distribute, sublicense,
-# and/or sell copies of the Software, and to permit persons to whom the
-# Software is furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-# DEALINGS IN THE SOFTWARE.
-
-from __future__ import print_function, with_statement
-
-import os
-import sys
-import re
-import string
-import gzip
-
-# Several "try" blocks for python2/3 differences (@secretrobotron)
-try:
-  import httplib
-except ImportError:
-  import http.client as httplib
-
-try:
-  import urlparse
-except ImportError:
-  import urllib.parse as urlparse
-
-try:
-  from BytesIO import BytesIO
-except ImportError:
-  from io import BytesIO
-
-try:
-  maketrans = str.maketrans
-except AttributeError:
-  maketrans = string.maketrans
-
-#
-# Begin
-#
-extPat = re.compile(r'^.*\.([A-Za-z]+)$')
-extDict = {
-  'html' : 'text/html',
-  'htm' : 'text/html',
-  'xhtml' : 'application/xhtml+xml',
-  'xht' : 'application/xhtml+xml',
-  'xml' : 'application/xml',
-}
-
-forceXml = False
-forceHtml = False
-gnu = False
-errorsOnly = False
-encoding = None
-fileName = None
-contentType = None
-inputHandle = None
-service = 'https://html5.validator.nu/'
-
-argv = sys.argv[1:]
-
-#
-# Parse command line input
-#
-for arg in argv:
-  if '--help' == arg:
-    print('-h : force text/html')
-    print('-x : force application/xhtml+xml')
-    print('-g : GNU output')
-    print('-e : errors only (no info or warnings)')
-    print('--encoding=foo : declare encoding foo')
-    print('--service=url  : the address of the HTML5 validator')
-    print('One file argument allowed. Leave out to read from stdin.')
-    sys.exit(0)
-  elif arg.startswith('--encoding='):
-    encoding = arg[11:]
-  elif arg.startswith('--service='):
-    service = arg[10:]
-  elif arg.startswith('--'):
-      sys.stderr.write('Unknown argument %s.\n' % arg)
-      sys.exit(2)
-  elif arg.startswith('-'):
-    for c in arg[1:]:
-      if 'x' == c:
-        forceXml = True
-      elif 'h' == c:
-        forceHtml = True
-      elif 'g' == c:
-        gnu = True
-      elif 'e' == c:
-        errorsOnly = True
-      else:
-        sys.stderr.write('Unknown argument %s.\n' % arg)
-        sys.exit(3)
-  else:
-    if fileName:
-      sys.stderr.write('Cannot have more than one input file.\n')
-      sys.exit(1)
-    fileName = arg
-
-#
-# Ensure a maximum of one forced output type
-#
-if forceXml and forceHtml:
-  sys.stderr.write('Cannot force HTML and XHTML at the same time.\n')
-  sys.exit(2)
-
-#
-# Set contentType
-#
-if forceXml:
-  contentType = 'application/xhtml+xml'
-elif forceHtml:
-  contentType = 'text/html'
-elif fileName:
-  m = extPat.match(fileName)
-  if m:
-    ext = m.group(1)
-    ext = ext.translate(maketrans(string.ascii_uppercase, string.ascii_lowercase))
-    if ext in extDict:
-      contentType = extDict[ext]
-    else:
-      sys.stderr.write('Unable to guess Content-Type from file name. Please force the type.\n')
-      sys.exit(3)
-  else:
-    sys.stderr.write('Could not extract a filename extension. Please force the type.\n')
-    sys.exit(6)
-else:
-  sys.stderr.write('Need to force HTML or XHTML when reading from stdin.\n')
-  sys.exit(4)
-
-if encoding:
-  contentType = '%s; charset=%s' % (contentType, encoding)
-
-#
-# Read the file argument (or STDIN)
-#
-if fileName:
-  inputHandle = fileName
-else:
-  inputHandle = sys.stdin
-
-with open(inputHandle, mode='rb') as inFile:
-  data = inFile.read()
-  with BytesIO() as buf:
-    # we could use another with block here, but it requires Python 2.7+
-    zipFile = gzip.GzipFile(fileobj=buf, mode='wb')
-    zipFile.write(data)
-    zipFile.close()
-    gzippeddata = buf.getvalue()
-
-#
-# Prepare the request
-#
-url = service
-
-if gnu:
-  url = url + '?out=gnu'
-else:
-  url = url + '?out=text'
-
-if errorsOnly:
-  url = url + '&level=error'
-
-connection = None
-response = None
-status = 302
-redirectCount = 0
-
-#
-# Make the request
-#
-while status in (302,301,307) and redirectCount < 10:
-  if redirectCount > 0:
-    url = response.getheader('Location')
-  parsed = urlparse.urlsplit(url)
-
-  if redirectCount > 0:
-    connection.close() # previous connection
-    print('Redirecting to %s' % url)
-    print('Please press enter to continue or type \'stop\' followed by enter to stop.')
-    if raw_input() != '':
-      sys.exit(0)
-
-  if parsed.scheme == 'https':
-    connection = httplib.HTTPSConnection(parsed[1])
-  else:
-    connection = httplib.HTTPConnection(parsed[1])
-
-  headers = {
-    'Accept-Encoding': 'gzip',
-    'Content-Type': contentType,
-    'Content-Encoding': 'gzip',
-    'Content-Length': len(gzippeddata),
-  }
-  urlSuffix = '%s?%s' % (parsed[2], parsed[3])
-
-  connection.connect()
-  connection.request('POST', urlSuffix, body=gzippeddata, headers=headers)
-
-  response = connection.getresponse()
-  status = response.status
-
-  redirectCount += 1
-
-#
-# Handle the response
-#
-if status != 200:
-  sys.stderr.write('%s %s\n' % (status, response.reason))
-  sys.exit(5)
-
-if response.getheader('Content-Encoding', 'identity').lower() == 'gzip':
-  response = gzip.GzipFile(fileobj=BytesIO(response.read()))
-
-if fileName and gnu:
-  quotedName = '"%s"' % fileName.replace("'", '\\042')
-  for line in response.read().split('\n'):
-    if line:
-      sys.stdout.write(quotedName)
-      sys.stdout.write(line + '\n')
-else:
-  output = response.read()
-  # python2/3 difference in output's type
-  if not isinstance(output, str):
-    output = output.decode('utf-8')
-  sys.stdout.write(output)
-
-connection.close()
--- a/tests/test_Attributes.py
+++ b/tests/test_Attributes.py
@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-

 from __future__ import with_statement
@ -6,10 +6,12 @@ from __future__ import unicode_literals

 import pytest
 import sys
-from sets import Set
 from ldapcherry.attributes import Attributes
 from ldapcherry.exceptions import *
 from ldapcherry.pyyamlwrapper import DumplicatedKey, RelationError
+if sys.version < '3':
+    from sets import Set as set
+

 class TestError(object):

@ -27,7 +29,7 @@ class TestError(object):
    def testGetSelfAttributes(self):
        inv = Attributes('./tests/cfg/attributes.yml')
        ret = inv.get_backends()
-        expected = Set(['ldap', 'ad'])
+        expected = set(['ldap', 'ad'])
        assert ret == expected

    def testGetSearchAttributes(self):
@ -40,6 +42,7 @@ class TestError(object):
        inv = Attributes('./tests/cfg/attributes.yml')
        ret = inv.get_backend_attributes('ldap')
        expected = ['shell', 'cn', 'userPassword', 'uidNumber', 'gidNumber', 'sn', 'home', 'givenName', 'email', 'uid']
+        expected.sort()
        assert ret == expected

    def testGetKey(self):
--- a/tests/test_BackendAD.py
+++ b/tests/test_BackendAD.py
@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-

 from __future__ import with_statement
@ -6,12 +6,14 @@ from __future__ import unicode_literals

 import pytest
 import sys
-from sets import Set
 from ldapcherry.backend.backendAD import Backend
 from ldapcherry.exceptions import *
 from disable import travis_disabled
 import cherrypy
 import logging
+if sys.version < '3':
+    from sets import Set as set
+

 cfg = {
    'display_name': u'test☭',
@ -74,25 +76,14 @@ class TestError(object):
        assert res == False

    @travis_disabled
-    def testMissingParam(self):
-        cfg2 = {}
-        return True
-        try:
-            inv = Backend(cfg2, cherrypy.log, u'test☭', attr, 'sAMAccountName')
-        except MissingKey:
-            return
-        else:
-            raise AssertionError("expected an exception")
-
-    @travis_disabled
-    def testSetPassword(self):
+    def testsetPassword(self):
        inv = Backend(cfg, cherrypy.log, u'test☭', attr, 'sAMAccountName')
        try:
            inv.add_user(default_user.copy())
            inv.add_to_groups(u'☭default_user', default_groups)
        except:
            pass
-	inv.set_attrs(u'☭default_user', {'unicodePwd': u'test☭P66642$'})
+        inv.set_attrs(u'☭default_user', {'unicodePwd': u'test☭P66642$'})
        ret = inv.auth(u'☭default_user', u'test☭P66642$')
        inv.del_user(u'☭default_user')
        assert ret == True
@ -128,16 +119,21 @@ class TestError(object):
    @travis_disabled
    def testSearchUser(self):
        inv = Backend(cfg, cherrypy.log, u'test☭', attr, 'sAMAccountName')
+        try:
+            inv.del_user(u'☭default_user')
+        except: pass
+        try:
+            inv.del_user(u'☭default_user2')
+        except: pass
        try:
            inv.add_user(default_user.copy())
-        except:
-            pass
+        except: pass
        inv.add_user(default_user2.copy())
        ret = inv.search(u'test☭')
        expected = [u'☭default_user', u'☭default_user2']
        inv.del_user(u'☭default_user')
        inv.del_user(u'☭default_user2')
-        assert Set(ret.keys()) == Set(expected)
+        assert set(ret.keys()) == set(expected)

    @travis_disabled
    def testAddUser(self):
--- a/tests/test_BackendDemo.py
+++ b/tests/test_BackendDemo.py
@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-

 from __future__ import with_statement
@ -6,12 +6,14 @@ from __future__ import unicode_literals

 import pytest
 import sys
-from sets import Set
 from ldapcherry.backend.backendDemo import Backend
 from ldapcherry.exceptions import *
 from disable import travis_disabled
 import cherrypy
 import logging
+if sys.version < '3':
+    from sets import Set as set
+

 cfg = {
    'display_name': 'test',
@ -69,16 +71,6 @@ class TestError(object):
        res = inv.auth('notauser', 'password') or inv.auth('default_user', 'notapassword')
        assert res == False

-    def testMissingParam(self):
-        cfg2 = {}
-        return True
-        try:
-            inv = Backend(cfg2, cherrypy.log, 'test', attr, 'uid')
-        except MissingKey:
-            return
-        else:
-            raise AssertionError("expected an exception")
-
    def testGetUser(self):
        inv = Backend(cfg, cherrypy.log, 'test', attr, 'uid')
        inv.add_user(default_user)
@ -92,7 +84,7 @@ class TestError(object):
        inv.add_user(default_user)
        inv.add_to_groups('default_user', default_groups)
        ret = inv.get_groups('default_user')
-        expected = Set(default_groups)
+        expected = set(default_groups)
        assert ret == expected

    def testSearchUser(self):
@ -101,7 +93,7 @@ class TestError(object):
        inv.add_user(default_user2)
        ret = inv.search('default')
        expected = ['default_user', 'default_user2']
-        assert Set(ret.keys()) == Set(expected)
+        assert set(ret.keys()) == set(expected)

    def testAddUser(self):
        try:
--- a/tests/test_BackendLdap.py
+++ b/tests/test_BackendLdap.py
@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-

 from __future__ import with_statement
@ -6,13 +6,14 @@ from __future__ import unicode_literals

 import pytest
 import sys
-from sets import Set
 from ldapcherry.backend.backendLdap import Backend, CaFileDontExist
 from ldapcherry.exceptions import *
 from disable import travis_disabled
 import cherrypy
 import logging
 import ldap
+if sys.version < '3':
+    from sets import Set as set

 cfg = {
 'module'             : 'ldapcherry.backend.ldap',
@ -104,7 +105,8 @@ class TestError(object):
        try:
            ldapc.simple_bind_s(inv.binddn, inv.bindpassword)
        except ldap.SERVER_DOWN as e:
-            assert e[0]['info'] == 'TLS: hostname does not match CN in peer certificate'
+            assert e.args[0]['info'] == 'TLS: hostname does not match CN in peer certificate' or \
+                    e.args[0]['info'] == '(unknown error code)'
        else:
            raise AssertionError("expected an exception")

@ -128,16 +130,6 @@ class TestError(object):
        res = inv.auth('notauser', 'password') or inv.auth(u'jwatsoné', 'notapasswordé')
        assert res == False

-    def testMissingParam(self):
-        cfg2 = {}
-        return True
-        try:
-            inv = Backend(cfg2, cherrypy.log, 'ldap', attr, 'uid')
-        except MissingKey:
-            return
-        else:
-            raise AssertionError("expected an exception")
-
    def testGetUser(self):
        inv = Backend(cfg, cherrypy.log, 'ldap', attr, 'uid')
        ret = inv.get_user(u'jwatsoné')
@ -158,7 +150,6 @@ class TestError(object):
        ]
        inv.add_to_groups(u'jwatsoné', groups)
        ret = inv.get_groups(u'jwatsoné')
-        print ret
        inv.del_from_groups(u'jwatsoné', ['cn=hrpeople,ou=Groups,dc=example,dc=org'])
        inv.del_from_groups(u'jwatsoné', ['cn=hrpeople,ou=Groups,dc=example,dc=org'])
        assert ret == ['cn=itpeople,ou=Groups,dc=example,dc=org', 'cn=hrpeople,ou=Groups,dc=example,dc=org']
--- a/tests/test_LdapCherry.py
+++ b/tests/test_LdapCherry.py
@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-

 from __future__ import with_statement
@ -9,7 +9,6 @@ import subprocess
 from tempfile import NamedTemporaryFile as tempfile
 import re

-from sets import Set
 from ldapcherry import LdapCherry
 from ldapcherry.exceptions import *
 from ldapcherry.pyyamlwrapper import DumplicatedKey, RelationError
@ -21,6 +20,9 @@ import logging
 from ldapcherry.lclogging import *
 from disable import *
 import json
+from tidylib import tidy_document
+if sys.version < '3':
+    from sets import Set as set

 cherrypy.session = {}

@ -78,19 +80,42 @@ class HtmlValidationFailed(Exception):
    def __init__(self, out):
        self.errors = out

+def _is_html_error(line):
+    for p in [
+                r'.*Warning: trimming empty <span>.*',
+                r'.*Error: <nav> is not recognized!.*',
+                r'.*Warning: discarding unexpected <nav>.*',
+                r'.*Warning: discarding unexpected </nav>.*',
+                r'.*Warning: <meta> proprietary attribute "charset".*',
+                r'.*Warning: <meta> lacks "content" attribute.*',
+                r'.*Warning: <link> inserting "type" attribute.*',
+                r'.*Warning: <link> proprietary attribute.*',
+                r'.*Warning: <input> proprietary attribute.*',
+                r'.*Warning: <button> proprietary attribute.*',
+                r'.*Warning: <form> proprietary attribute.*',
+                r'.*Warning: <table> lacks "summary" attribute.*',
+                r'.*Warning: <script> inserting "type" attribute.*',
+                r'.*Warning: <input> attribute "id" has invalid value.*',
+                r'.*Warning: <a> proprietary attribute.*',
+                r'.*Warning: <input> attribute "type" has invalid value.*',
+                r'.*Warning: <span> proprietary attribute.*',
+             ]:
+        if re.match(p, line):
+            return False
+    return True
+
 def htmlvalidator(page):
+    document, errors = tidy_document(page,
+        options={'numeric-entities':1})
    f = tempfile()
-    stdout = tempfile()
-    f.write(page.encode("utf-8"))
-    f.seek(0)
-    ret = subprocess.call(['./tests/html_validator.py', '-h', f.name], stdout=stdout)
-    stdout.seek(0)
-    out = stdout.read()
-    f.close()
-    stdout.close()
-    print(out)
-    if not re.search(r'Error:.*', out) is None:
-        raise HtmlValidationFailed(out)
+    for line in errors.splitlines():
+        if _is_html_error(line):
+            print("################")
+            print("Blocking error: '%s'" % line)
+            print("all tidy_document errors:")
+            print(errors)
+            print("################")
+            raise HtmlValidationFailed(line)

 class BadModule():
    pass
@ -127,7 +152,7 @@ class TestError(object):
    def testLog(self):
        app = LdapCherry()
        cfg = { 'global' : {}}
-        for t in ['none', 'file', 'syslog']:
+        for t in ['none', 'file', 'syslog', 'stdout']:
            cfg['global']['log.access_handler']=t
            cfg['global']['log.error_handler']=t
            app._set_access_log(cfg, logging.DEBUG)
@ -184,7 +209,7 @@ class TestError(object):
            app.login(u'jwatsoné', u'passwordwatsoné')
        except cherrypy.HTTPRedirect as e:
            expected = 'http://127.0.0.1:8080/'
-            assert e[0][0] == expected
+            assert e.urls[0] == expected
        else:
            raise AssertionError("expected an exception")

@ -196,7 +221,7 @@ class TestError(object):
            app.login(u'jwatsoné', u'wrongPasswordé')
        except cherrypy.HTTPRedirect as e:
            expected = 'http://127.0.0.1:8080/signin'
-            assert e[0][0] == expected
+            assert e.urls[0] == expected
        else:
            raise AssertionError("expected an exception")

@ -337,7 +362,7 @@ class TestError(object):
        app = LdapCherry()
        loadconf('./tests/cfg/ldapcherry_adldap.cfg', app)
        inv = ldapcherry.backend.backendAD.Backend(adcfg, cherrypy.log, u'test☭', adattr, 'sAMAccountName')
-	try:
+        try:
            app._deleteuser(u'☭default_user')
        except:
            pass
@ -360,4 +385,3 @@ class TestError(object):
        get_loglevel('alert') is logging.CRITICAL and \
        get_loglevel('emergency') is logging.CRITICAL and \
        get_loglevel('notalevel') is logging.INFO
-
--- a/tests/test_Roles.py
+++ b/tests/test_Roles.py
@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # -*- coding: utf-8 -*-

 from __future__ import with_statement
@ -6,16 +6,17 @@ from __future__ import unicode_literals

 import pytest
 import sys
-from sets import Set
 from ldapcherry.roles import Roles
 from ldapcherry.exceptions import DumplicateRoleKey, MissingKey, DumplicateRoleContent, MissingRolesFile, MissingRole
 from ldapcherry.pyyamlwrapper import DumplicatedKey, RelationError
+if sys.version < '3':
+    from sets import Set as set

 class TestError(object):

    def testNominal(self):
        inv = Roles('./tests/cfg/roles.yml')
-        print inv.roles
+        print(inv.roles)
        return True

    def testMissingDisplayName(self):
@ -64,7 +65,7 @@ class TestError(object):
                ['admin-lv2', 'admin-lv3', 'users'],
                ['admin-lv2']
        )
-        expected = {'ad': Set(['Administrators', 'Domain Controllers']), 'ldap': Set(['cn=nagios admins,ou=group,dc=example,dc=com', 'cn=puppet admins,ou=group,dc=example,dc=com', 'cn=dns admins,ou=group,dc=example,dc=com'])}
+        expected = {'ad': set(['Administrators', 'Domain Controllers']), 'ldap': set(['cn=nagios admins,ou=group,dc=example,dc=com', 'cn=puppet admins,ou=group,dc=example,dc=com', 'cn=dns admins,ou=group,dc=example,dc=com'])}
        assert groups == expected

    def testGetGroup(self):
@ -78,7 +79,50 @@ class TestError(object):

    def testNested(self):
        inv = Roles('./tests/cfg/nested.yml')
-        expected = {'developpers': {'backends_groups': {'ad': ['Domain Users'], 'ldap': ['cn=developpers,ou=group,dc=example,dc=com', 'cn=users,ou=group,dc=example,dc=com']}, 'display_name': 'Developpers', 'description': 'description'}, 'admin-lv3': {'backends_groups': {'ad': ['Domain Users', 'Administrators', 'Domain Controllers'], 'ldap': ['cn=nagios admins,ou=group,dc=example,dc=com', 'cn=users,ou=group,dc=example,dc=com', 'cn=puppet admins,ou=group,dc=example,dc=com', 'cn=dns admins,ou=group,dc=example,dc=com']}, 'display_name': 'Administrators Level 3', 'description': 'description'}, 'admin-lv2': {'backends_groups': {'ad': ['Domain Users'], 'ldap': ['cn=nagios admins,ou=group,dc=example,dc=com', 'cn=users,ou=group,dc=example,dc=com']}, 'display_name': 'Administrators Level 2', 'description': 'description', 'LC_admins': True}, 'users': {'backends_groups': {'ad': ['Domain Users'], 'ldap': ['cn=users,ou=group,dc=example,dc=com']}, 'display_name': 'Simple Users', 'description': 'description'}}
+        expected = {
+            'admin-lv2': {
+                'LC_admins': True,
+                'backends_groups': {
+                    'ad': ['Domain Users'],
+                     'ldap': ['cn=nagios '
+                              'admins,ou=group,dc=example,dc=com',
+                              'cn=users,ou=group,dc=example,dc=com']
+                },
+                'description': 'description',
+                'display_name': 'Administrators Level 2'
+            },
+            'admin-lv3': {
+                'backends_groups': {
+                    'ad': ['Administrators',
+                           'Domain Controllers',
+                           'Domain Users'],
+                    'ldap': ['cn=dns '
+                             'admins,ou=group,dc=example,dc=com',
+                             'cn=nagios '
+                             'admins,ou=group,dc=example,dc=com',
+                             'cn=puppet '
+                             'admins,ou=group,dc=example,dc=com',
+                             'cn=users,ou=group,dc=example,dc=com']
+                    },
+                'description': 'description',
+                'display_name': 'Administrators Level 3'
+            },
+            'developers': {
+                'backends_groups': {
+                    'ad': ['Domain Users'],
+                    'ldap': ['cn=developers,ou=group,dc=example,dc=com',
+                             'cn=users,ou=group,dc=example,dc=com']},
+                'description': 'description',
+                'display_name': 'Developpers'
+            },
+            'users': {
+                'backends_groups': {
+                    'ad': ['Domain Users'],
+                    'ldap': ['cn=users,ou=group,dc=example,dc=com']},
+            'description': 'description',
+            'display_name': 'Simple Users'
+            }
+        }
        assert expected == inv.flatten

    def testGetGroupMissingRole(self):
@ -108,13 +152,13 @@ class TestError(object):
    def testGetAllRoles(self):
        inv = Roles('./tests/cfg/roles.yml')
        res = inv.get_allroles()
-        expected = ['developpers', 'admin-lv3', 'admin-lv2', 'users']
+        expected = ['developers', 'admin-lv3', 'admin-lv2', 'users']
        assert res == expected

    def testGetAllRoles(self):
        inv = Roles('./tests/cfg/roles.yml')
        res = inv.get_backends()
-        expected = Set(['ad', 'ldap'])
+        expected = set(['ad', 'ldap'])
        assert res == expected

    def testDumpNested(self):
@ -143,9 +187,9 @@ class TestError(object):
        'ad' : ['Domain Users', 'Domain Users 2'],
        'ldap': ['cn=users,ou=group,dc=example,dc=com',
            'cn=nagios admins,ou=group,dc=example,dc=com',
-            'cn=developpers,ou=group,dc=example,dc=com',
+            'cn=developers,ou=group,dc=example,dc=com',
            ],
        'toto': ['not a group'],
        }
-        expected = {'unusedgroups': {'toto': Set(['not a group']), 'ad': Set(['Domain Users 2'])}, 'roles': Set(['developpers', 'admin-lv2', 'users'])}
+        expected = {'unusedgroups': {'toto': set(['not a group']), 'ad': set(['Domain Users 2'])}, 'roles': set(['developers', 'admin-lv2', 'users'])}
        assert inv.get_roles(groups) == expected
--- a/tests/test_env/deploy.sh
+++ b/tests/test_env/deploy.sh
@ -1,19 +1,27 @@
 #!/bin/sh

-DEBIAN_FRONTEND=noninteractive apt-get install ldap-utils slapd -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold"  -f -q -y
-DEBIAN_FRONTEND=noninteractive apt-get install samba -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold"  -f -q -y
-DEBIAN_FRONTEND=noninteractive apt-get install winbind -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold"  -f -q -y
+apt update
+
+DEBIAN_FRONTEND=noninteractive apt-get install ldap-utils slapd -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold"  -f -q -y
+DEBIAN_FRONTEND=noninteractive apt-get install samba-dsdb-modules samba-vfs-modules samba -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold"  -f -q -y
+DEBIAN_FRONTEND=noninteractive apt-get install winbind -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold"  -f -q -y
+DEBIAN_FRONTEND=noninteractive apt-get install build-essential python3-dev libsasl2-dev slapd ldap-utils tox lcov valgrind libtidy-dev libldap-dev python3-cherrypy3 python3-ldap python3-mako -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold"  -f -q -y
+
+[ -e '/etc/default/slapd' ] && rm -rf /etc/default/slapd
+cp -r `dirname $0`/etc/default/slapd /etc/default/slapd
+[ -e '/etc/ldap' ] && rm -rf /etc/ldap
+cp -r `dirname $0`/etc/ldap /etc/ldap
+[ -e '/etc/ldapcherry' ] && rm -rf /etc/ldapcherry
+cp -r `dirname $0`/etc/ldapcherry /etc/ldapcherry

-rsync -a `dirname $0`/ /
 cd `dirname $0`/../../
 sudo sed -i "s%template_dir.*%template_dir = '`pwd`/resources/templates/'%" /etc/ldapcherry/ldapcherry.ini
 sudo sed -i "s%tools.staticdir.dir.*%tools.staticdir.dir = '`pwd`/resources/static/'%" /etc/ldapcherry/ldapcherry.ini

 chown -R openldap:openldap /etc/ldap/
-rm /etc/ldap/slapd.d/cn\=config/*mdb*
 /etc/init.d/slapd restart
 ldapadd -c -H ldap://localhost:390  -x -D "cn=admin,dc=example,dc=org" -f /etc/ldap/content.ldif -w password
-if grep -q '127.0.0.1' /etc/hosts
+if grep -q '127.0.0.1' /etc/hosts && ! grep -q 'ldap.ldapcherry.org' /etc/hosts
 then
    sed -i "s/\(127.0.0.1.*\)/\1 ldap.ldapcherry.org ad.ldapcherry.org ldap.dnscherry.org/" /etc/hosts
 else
@ -24,11 +32,6 @@ cat /etc/hosts

 df -h

-/etc/init.d/samba stop 
-/etc/init.d/smbd stop
-/etc/init.d/nmbd stop 
-/etc/init.d/samba-ad-dc stop
-
 find /var/log/samba/ -type f -exec rm -f {} \;

 smbconffile=/etc/samba/smb.conf
@ -40,9 +43,11 @@ role=dc
 sambacmd=samba-tool
 adpass=qwertyP455

+systemctl unmask samba-ad-dc
+
 hostname ad.ldapcherry.org 
-/etc/init.d/dnsmasq stop
 pkill -9 dnsmasq
+pkill -9 samba

 kill -9 `cat /var/run/samba/smbd.pid` 
 rm -f /var/run/samba/smbd.pid
@ -65,12 +70,16 @@ cat ${smbconffile}

 mv /var/lib/samba/private/krb5.conf /etc/krb5.conf

-sleep 5
+sleep 15

-/etc/init.d/samba-ad-dc start
+systemctl restart samba-ad-dc
+/etc/init.d/samba-ad-dc restart

 cat /var/log/samba/*

 sleep 5

-netstat -apn | grep samba
+samba-tool domain passwordsettings set -d 1 --complexity off
+samba-tool domain passwordsettings set -d 1 --min-pwd-length 0
+systemctl status samba-ad-dc
+ss -apn | grep samba
--- a/tests/test_env/etc/ldap/content.ldif
+++ b/tests/test_env/etc/ldap/content.ldif
@ -103,9 +103,9 @@ cn: users
 description: Basic Users
 member: cn=Sheri Smith,ou=people,dc=example,dc=org

-dn: cn=developpers,ou=group,dc=example,dc=org
+dn: cn=developers,ou=group,dc=example,dc=org
 objectclass: groupofnames
-cn: developpers
+cn: developers
 description: Developpers 
 member: cn=Sheri Smith,ou=people,dc=example,dc=org

--- a/tests/test_env/etc/ldap/slapd.d/cn=config.ldif
+++ b/tests/test_env/etc/ldap/slapd.d/cn=config.ldif
@ -1,4 +1,5 @@
 # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify.
+# CRC32 7a6099db
 dn: cn=config
 objectClass: olcGlobal
 cn: config
@ -7,12 +8,12 @@ olcLogLevel: none
 olcPidFile: /var/run/slapd/slapd.pid
 olcToolThreads: 1
 structuralObjectClass: olcGlobal
-entryUUID: 2964261c-6754-1033-8d48-1703270f04bd
+entryUUID: 38579c70-750a-103e-8489-9578878139e2
 creatorsName: cn=config
-createTimestamp: 20140503211805Z
-entryCSN: 20140503211805.658522Z#000000#000#000000
+createTimestamp: 20240312221838Z
+entryCSN: 20240312221838.644900Z#000000#000#000000
 modifiersName: cn=config
-modifyTimestamp: 20140503211805Z
+modifyTimestamp: 20240312221838Z
 olcTLSCACertificateFile: /etc/ldap/ssl/TEST-cacert.pem
 olcTLSCertificateFile: /etc/ldap/ssl/ldap@dnscherry.org-cert.pem
 olcTLSCertificateKeyFile: /etc/ldap/ssl/ldap@dnscherry.org-key.pem
--- a/tests/test_env/etc/ldap/slapd.d/cn=config/cn=module{0}.ldif
+++ b/tests/test_env/etc/ldap/slapd.d/cn=config/cn=module{0}.ldif
@ -1,14 +1,14 @@
 # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify.
-# CRC32 d8758c75
+# CRC32 baf86138
 dn: cn=module{0}
 objectClass: olcModuleList
 cn: module{0}
 olcModulePath: /usr/lib/ldap
-olcModuleLoad: {0}back_hdb
+olcModuleLoad: {0}back_mdb
 structuralObjectClass: olcModuleList
-entryUUID: 2964fd58-6754-1033-8d50-1703270f04bd
+entryUUID: 3857cd3a-750a-103e-8491-9578878139e2
 creatorsName: cn=admin,cn=config
-createTimestamp: 20140503211805Z
-entryCSN: 20140503211805.664149Z#000000#000#000000
+createTimestamp: 20240312221838Z
+entryCSN: 20240312221838.646168Z#000000#000#000000
 modifiersName: cn=admin,cn=config
-modifyTimestamp: 20140503211805Z
+modifyTimestamp: 20240312221838Z
--- a/tests/test_env/etc/ldap/slapd.d/cn=config/cn=schema.ldif
+++ b/tests/test_env/etc/ldap/slapd.d/cn=config/cn=schema.ldif
@ -1,12 +1,12 @@
 # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify.
-# CRC32 db8c9607
+# CRC32 fff51a11
 dn: cn=schema
 objectClass: olcSchemaConfig
 cn: schema
 structuralObjectClass: olcSchemaConfig
-entryUUID: 29644a02-6754-1033-8d4b-1703270f04bd
+entryUUID: 3857a42c-750a-103e-848c-9578878139e2
 creatorsName: cn=admin,cn=config
-createTimestamp: 20140503211805Z
-entryCSN: 20140503211805.659557Z#000000#000#000000
+createTimestamp: 20240312221838Z
+entryCSN: 20240312221838.645118Z#000000#000#000000
 modifiersName: cn=admin,cn=config
-modifyTimestamp: 20140503211805Z
+modifyTimestamp: 20240312221838Z
--- a/tests/test_env/etc/ldap/slapd.d/cn=config/olcBackend={0}hdb.ldif
+++ b/tests/test_env/etc/ldap/slapd.d/cn=config/olcBackend={0}hdb.ldif
@ -1,12 +0,0 @@
-# AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify.
-# CRC32 94d41de9
-dn: olcBackend={0}hdb
-objectClass: olcBackendConfig
-olcBackend: {0}hdb
-structuralObjectClass: olcBackendConfig
-entryUUID: 2965a6f4-6754-1033-8d51-1703270f04bd
-creatorsName: cn=admin,cn=config
-createTimestamp: 20140503211805Z
-entryCSN: 20140503211805.668489Z#000000#000#000000
-modifiersName: cn=admin,cn=config
-modifyTimestamp: 20140503211805Z
--- a/tests/test_env/etc/ldap/slapd.d/cn=config/olcDatabase={-1}frontend.ldif
+++ b/tests/test_env/etc/ldap/slapd.d/cn=config/olcDatabase={-1}frontend.ldif
@ -1,5 +1,5 @@
 # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify.
-# CRC32 511a173e
+# CRC32 e8f0a13c
 dn: olcDatabase={-1}frontend
 objectClass: olcDatabaseConfig
 objectClass: olcFrontendConfig
@ -10,9 +10,9 @@ olcAccess: {1}to dn.exact="" by * read
 olcAccess: {2}to dn.base="cn=Subschema" by * read
 olcSizeLimit: 500
 structuralObjectClass: olcDatabaseConfig
-entryUUID: 296430d0-6754-1033-8d49-1703270f04bd
+entryUUID: 38579ebe-750a-103e-848a-9578878139e2
 creatorsName: cn=config
-createTimestamp: 20140503211805Z
-entryCSN: 20140503211805.658911Z#000000#000#000000
+createTimestamp: 20240312221838Z
+entryCSN: 20240312221838.644978Z#000000#000#000000
 modifiersName: cn=config
-modifyTimestamp: 20140503211805Z
+modifyTimestamp: 20240312221838Z
--- a/tests/test_env/etc/ldap/slapd.d/cn=config/olcDatabase={0}config.ldif
+++ b/tests/test_env/etc/ldap/slapd.d/cn=config/olcDatabase={0}config.ldif
@ -1,5 +1,5 @@
 # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify.
-# CRC32 fc130d0e
+# CRC32 caed0a01
 dn: olcDatabase={0}config
 objectClass: olcDatabaseConfig
 olcDatabase: {0}config
@ -7,9 +7,9 @@ olcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external
 ,cn=auth manage by * break
 olcRootDN: cn=admin,cn=config
 structuralObjectClass: olcDatabaseConfig
-entryUUID: 29644142-6754-1033-8d4a-1703270f04bd
+entryUUID: 3857a274-750a-103e-848b-9578878139e2
 creatorsName: cn=config
-createTimestamp: 20140503211805Z
-entryCSN: 20140503211805.659332Z#000000#000#000000
+createTimestamp: 20240312221838Z
+entryCSN: 20240312221838.645074Z#000000#000#000000
 modifiersName: cn=config
-modifyTimestamp: 20140503211805Z
+modifyTimestamp: 20240312221838Z
--- a/tests/test_env/etc/ldap/slapd.d/cn=config/olcDatabase={1}mdb.ldif
+++ b/tests/test_env/etc/ldap/slapd.d/cn=config/olcDatabase={1}mdb.ldif
@ -1,7 +1,9 @@
-dn: olcDatabase={1}hdb
+# AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify.
+# CRC32 91d29b33
+dn: olcDatabase={1}mdb
 objectClass: olcDatabaseConfig
-objectClass: olcHdbConfig
-olcDatabase: {1}hdb
+objectClass: olcMdbConfig
+olcDatabase: {1}mdb
 olcDbDirectory: /var/lib/ldap
 olcSuffix: dc=example,dc=org
 olcAccess: {0}to attrs=userPassword,shadowLastChange by self write by anonymou
@ -13,15 +15,15 @@ olcLastMod: TRUE
 olcRootDN: cn=admin,dc=example,dc=org
 olcRootPW: {SSHA}Fp+rSxe5eFsj0DGITJts4DwdSDFDZG9P
 olcDbCheckpoint: 512 30
-olcDbConfig: {0}set_cachesize 0 2097152 0
-olcDbConfig: {1}set_lk_max_objects 1500
-olcDbConfig: {2}set_lk_max_locks 1500
-olcDbConfig: {3}set_lk_max_lockers 1500
 olcDbIndex: objectClass eq
-structuralObjectClass: olcHdbConfig
-entryUUID: 2965af5a-6754-1033-8d52-1703270f04bd
+olcDbIndex: cn,uid eq
+olcDbIndex: uidNumber,gidNumber eq
+olcDbIndex: member,memberUid eq
+olcDbMaxSize: 1073741824
+structuralObjectClass: olcMdbConfig
+entryUUID: 3857d7ee-750a-103e-8492-9578878139e2
 creatorsName: cn=admin,cn=config
-createTimestamp: 20140503211805Z
-entryCSN: 20140503211805.668708Z#000000#000#000000
+createTimestamp: 20240312221838Z
+entryCSN: 20240312221838.646442Z#000000#000#000000
 modifiersName: cn=admin,cn=config
-modifyTimestamp: 20140503211805Z
+modifyTimestamp: 20240312221838Z
--- a/tests/test_env/etc/ldapcherry/attributes.yml
+++ b/tests/test_env/etc/ldapcherry/attributes.yml
@ -0,0 +1,132 @@
+cn:
+    description: "First Name and Display Name"
+    display_name: "Display Name"
+    type: string
+    weight: 30
+    autofill: 
+        function: lcDisplayName
+        args:
+            - $first-name
+            - $name
+    backends:
+        ldap: cn
+        ad: cn
+first-name:
+    description: "First name of the user"
+    display_name: "First Name"
+    search_displayed: True
+    type: string
+    weight: 20
+    backends:
+        ldap: givenName
+        ad: givenName
+name:
+    description: "Family name of the user"
+    display_name: "Name"
+    search_displayed: True
+    weight: 10
+    type: string
+    backends:
+        ldap: sn
+        ad: sn
+email:
+    description: "Email of the user"
+    display_name: "Email"
+    search_displayed: True
+    type: email
+    weight: 40
+    autofill: 
+        function: lcMail
+        args:
+            - $first-name
+            - $name
+            - '@example.com'
+    backends:
+        ldap: mail
+        ad: mail
+uid:
+    description: "UID of the user"
+    display_name: "UID"
+    search_displayed: True
+    key: True
+    type: string
+    weight: 50
+    autofill: 
+        function: lcUid
+        args:
+            - $first-name
+            - $name
+            - '10000'
+            - '40000'
+    backends:
+        ldap: uid
+        ad: sAMAccountName
+uidNumber:
+    description: "User ID Number of the user"
+    display_name: "UID Number"
+    weight: 60
+    type: int
+    autofill: 
+        function: lcUidNumber
+        args:
+            - $first-name
+            - $name
+            - '10000'
+            - '40000'
+    backends:
+        ldap: uidNumber
+        ad: uidNumber
+gidNumber:
+    description: "Group ID Number of the user"
+    display_name: "GID Number"
+    weight: 70
+    type: int
+    default: '10000'
+    backends:
+        ldap: gidNumber
+        ad: gidNumber
+shell:
+    description: "Shell of the user"
+    display_name: "Shell"
+    weight: 80
+    self: True
+    type: stringlist
+    values:
+        - /bin/bash
+        - /bin/zsh
+        - /bin/sh
+    backends:
+        ldap: loginShell
+        ad: loginShell 
+home:
+    description: "Home user path"
+    display_name: "Home"
+    weight: 90
+    type: string
+    autofill: 
+        function: lcHomeDir
+        args:
+            - $first-name
+            - $name
+            - /home/
+    backends:
+        ldap: homeDirectory
+        ad: homeDirectory
+password:
+    description: "Password of the user"
+    display_name: "Password"
+    weight: 31
+    self: True
+    type: password
+    backends:
+        ldap: userPassword
+        ad: unicodePwd
+
+#logscript:
+#    description: "Windows login script"
+#    display_name: "Login script"
+#    weight: 100
+#    type: fix
+#    value: login1.bat
+#    backends:
+#        ad: scriptPath
--- a/tests/test_env/etc/ldapcherry/ldapcherry.ini
+++ b/tests/test_env/etc/ldapcherry/ldapcherry.ini
@ -0,0 +1,125 @@
+# global parameters
+[global]
+
+# listing interface
+server.socket_host = '127.0.0.1'
+# port
+server.socket_port = 8080
+# number of threads
+server.thread_pool = 8
+#don't show traceback on error
+request.show_tracebacks = False
+
+# log configuration
+# /!\ you can't have multiple log handlers
+#####################################
+#   configuration to log in files   #
+#####################################
+## logger 'file' for access log 
+#log.access_handler = 'file'
+## logger syslog for error and ldapcherry log 
+#log.error_handler = 'file'
+## access log file
+#log.access_file = '/tmp/ldapcherry_access.log'
+## error and ldapcherry log file
+#log.error_file = '/tmp/ldapcherry_error.log'
+
+#####################################
+#  configuration to log in syslog   #
+#####################################
+# logger syslog for access log 
+#log.access_handler = 'syslog'
+## logger syslog for error and ldapcherry log 
+log.error_handler = 'syslog'
+
+#####################################
+#  configuration to not log at all  #
+#####################################
+# logger none for access log 
+log.access_handler = 'syslog'
+# logger none for error and ldapcherry log 
+#log.error_handler = 'none'
+
+# log level
+log.level = 'debug'
+
+# session configuration
+# activate session
+tools.sessions.on = True
+# session timeout
+tools.sessions.timeout = 10
+# file session storage(to use if multiple processes, 
+# default is in RAM and per process)
+#tools.sessions.storage_type = "file"
+# session 
+#tools.sessions.storage_path = "/var/lib/ldapcherry/sessions"
+
+[attributes]
+
+# file discribing form content
+attributes.file = '/etc/ldapcherry/attributes.yml'
+
+[roles]
+
+# file listing roles
+roles.file = '/etc/ldapcherry/roles.yml'
+
+[backends]
+
+ldap.module = 'ldapcherry.backend.backendLdap'
+ldap.groupdn = 'ou=Group,dc=example,dc=org'
+ldap.userdn = 'ou=people,dc=example,dc=org'
+ldap.binddn = 'cn=dnscherry,dc=example,dc=org'
+ldap.password = 'password'
+ldap.uri = 'ldap://ldap.ldapcherry.org:390'
+ldap.ca = '/etc/dnscherry/TEST-cacert.pem'
+ldap.starttls = 'off'
+ldap.checkcert = 'off'
+ldap.user_filter_tmpl = '(uid=%(username)s)'
+ldap.group_filter_tmpl = '(member=%(userdn)s)'
+ldap.search_filter_tmpl = '(|(uid=%(searchstring)s*)(sn=%(searchstring)s*))'
+ldap.group_attr.member = "%(dn)s"
+
+#ldap.objectclasses = 'top, person, organizationalPerson, user'
+ldap.objectclasses = 'top, person, posixAccount, inetOrgPerson'
+ldap.dn_user_attr = 'uid'
+ldap.timeout = 1
+
+ad.module = 'ldapcherry.backend.backendAD'
+ad.domain = 'dc.ldapcherry.org'
+ad.login  = 'administrator'
+ad.password = 'qwertyP455'
+ad.uri = 'ldaps://ldap.ldapcherry.org:636'
+ad.checkcert = 'off'
+
+# authentification parameters
+[auth]
+
+# Auth mode
+# * and: user must authenticate on all backends
+# * or:  user must authenticate on one of the backend
+# * none: disable authentification
+# * custom: custom authentification module (need auth.module param)
+auth.mode = 'none'
+
+# custom auth module to load
+#auth.module = 'ldapcherry.auth.modNone'
+
+[ppolicy]
+
+# password policy module
+ppolicy.module = 'ldapcherry.ppolicy.simple'
+
+# parameters of the module
+min_length = 2
+min_upper = 0
+min_digit = 0
+
+# resources parameters
+[resources]
+# templates directory
+templates.dir = './resources/templates/'
+
+[/static]
+tools.staticdir.on = True
+tools.staticdir.dir = './resources/static/'
--- a/tests/test_env/etc/ldapcherry/roles.yml
+++ b/tests/test_env/etc/ldapcherry/roles.yml
@ -0,0 +1,36 @@
+admin-lv3:
+    display_name: Administrators Level 3
+    description: description
+    backends_groups:
+        ldap:
+            - cn=dns admins,ou=Group,dc=example,dc=org
+            - cn=nagios admins,ou=Group,dc=example,dc=org
+            - cn=puppet admins,ou=Group,dc=example,dc=org
+            - cn=users,ou=Group,dc=example,dc=org
+        ad:
+            - Administrators
+            - Domain Controllers 
+
+admin-lv2:
+    display_name: Administrators Level 2
+    description: description
+    LC_admins: True
+    backends_groups:
+        ldap:
+            - cn=nagios admins,ou=Group,dc=example,dc=org
+            - cn=users,ou=Group,dc=example,dc=org
+
+developers:
+    display_name: Developpers
+    description: description
+    backends_groups:
+        ldap:
+            - cn=developers,ou=Group,dc=example,dc=org
+            - cn=users,ou=Group,dc=example,dc=org
+
+users:
+    display_name: Simple Users
+    description: description
+    backends_groups:
+        ldap:
+            - cn=users,ou=Group,dc=example,dc=org
Author	SHA1	Message	Date
Carpentier Pierre-Francois	dd331f948c	Update README.rst	2024-03-13 01:13:02 +01:00
Carpentier Pierre-Francois	a2e985576b	Update README.rst	2024-03-13 00:57:26 +01:00
kakwa	6a2e6e56d0	fix CI badge and remove the coverage one	2024-03-13 00:35:48 +01:00
kakwa	5d8fc08f5b	re-enable ssl/tls	2024-03-13 00:29:50 +01:00
kakwa	59855f0090	just rename stuff	2024-03-13 00:21:04 +01:00
kakwa	d1215bc56f	fix package name	2024-03-13 00:18:47 +01:00
kakwa	ef11e6a7e3	more deps	2024-03-13 00:17:33 +01:00
kakwa	3596e14249	adding stuff to install	2024-03-13 00:13:11 +01:00
kakwa	b94713acb6	sudo for CI	2024-03-13 00:10:06 +01:00
kakwa	de16df475f	add basic gh action tests	2024-03-13 00:06:44 +01:00
kakwa	242c9ab96e	update test ldap configuration	2024-03-13 00:00:14 +01:00
kakwa	48dbff983d	update shebang	2024-03-13 00:00:06 +01:00
kakwa	6413b782dd	update shebang	2024-03-12 23:59:46 +01:00
Carpentier Pierre-Francois	55c49b4eff	Merge pull request #68 from kakwa/dependabot/pip/python-ldap-3.4.0 Bump python-ldap from 2.4.15 to 3.4.0	2024-03-12 22:39:47 +01:00
Carpentier Pierre-Francois	53a4b6fd5e	Merge pull request #59 from cyberb/master lcCopy	2024-03-12 22:36:33 +01:00
dependabot[bot]	0ccbe95d9a	Bump python-ldap from 2.4.15 to 3.4.0 Bumps [python-ldap](https://github.com/python-ldap/python-ldap) from 2.4.15 to 3.4.0. - [Release notes](https://github.com/python-ldap/python-ldap/releases) - [Commits](https://github.com/python-ldap/python-ldap/commits/python-ldap-3.4.0) --- updated-dependencies: - dependency-name: python-ldap dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>	2021-11-29 18:00:11 +00:00
Carpentier Pierre-Francois	4da050236d	Merge pull request #49 from smacz42/andrewcz-ldapcherry-issue34 Add demo backend configuration files in goodies	2020-05-20 19:24:55 +02:00
Carpentier Pierre-Francois	135699bd48	Merge pull request #62 from Sohalt/master fix typo	2020-04-07 20:58:09 +02:00
sohalt	1549a172f7	fix typo	2020-04-07 19:15:31 +02:00
Boris Rybalkin	e2ab3e85d8	lcCopy	2020-01-10 22:10:28 +00:00
Carpentier Pierre-Francois	b5907ce340	Merge pull request #55 from StbX/master add missing import	2019-10-23 21:09:28 +02:00
Steffen Brandemann	791895d4c3	add missing os import	2019-10-21 23:17:20 +02:00
AndrewCz	b12add4a0f	Add demo backend configuration files in goodies	2019-07-21 20:06:19 -04:00
Carpentier Pierre-Francois	856157af79	Merge pull request #24 from jqueuniet/fix_default_handler Fix default handler arguments	2019-05-03 12:09:13 +02:00
Carpentier Pierre-Francois	b5e7cb6a44	Merge pull request #33 from smacz42/install-in-readme Add install command to setup in README	2019-04-23 19:56:51 +02:00
smacz	d61c89460e	Add install command to setup in README	2019-04-23 11:24:17 -04:00
Johann Queuniet	0a96ca61d5	Fix default handler arguments	2019-03-26 11:05:39 +01:00
Carpentier Pierre-Francois	3b58f1464e	Merge pull request #23 from jqueuniet/fix_email_regexp Change the email regular expression to fix a few validation issues	2019-03-20 18:33:19 +01:00
Johann Queuniet	50c6259035	Fix email regexp	2019-03-20 15:57:45 +01:00
kakwa	245bafb01c	typo	2019-02-13 09:44:47 +01:00
kakwa	5ee8a74040	update documentation	2019-02-13 09:41:17 +01:00
kakwa	96acda7aa6	fix formatting	2019-02-13 09:19:08 +01:00
kakwa	30c28c5feb	slightly more robust unit tests pre-cleanup to avoid contamination if other tests fail	2019-02-12 23:28:52 +01:00
kakwa	dc60300a29	version + changelog	2019-02-12 23:28:41 +01:00
kakwa	882a303474	fix crash due to encoding in python 2	2019-02-12 23:27:30 +01:00
kakwa	d831b09293	improve documentation * improve documentation for key: True flag in attributes.yml * improve documentation for the ldap filters and their templating * improve comment in the .ini file	2019-02-12 23:06:42 +01:00
kakwa	7ac7118c9a	adding a debug log to help figure out issues with filters.	2019-02-12 22:43:03 +01:00
kakwa	d690bbdc41	passing the correct logger to the backend previously, the default logger was passed, this logger was using the default configuration and log level, not honoring log level in particular. As a consequence, it was impossible to get debug logs from the backend. This is now working as expected.	2019-02-12 22:40:42 +01:00
kakwa	73c02ccff4	disable default logger if running in debug mode (-D) Debug mode can lead to log sensitive data (like password changes). Having -D to log exclusively to stderr is a safer option.	2019-02-12 22:36:16 +01:00
kakwa	799ca2403f	fix the urls for modify and delete The id of the user is passed through the querystring in this page. But the id was not properly escaped to be included as a querystring parameter leading to weird issues like.	2019-02-12 21:24:24 +01:00
kakwa	bbafafae60	remove the double escaping. Now the escaping is done by in the templates. We need to remove the previous escaping done by hand in the code. Otherwise, we end-up with double escaping and funky displaying of fields.	2019-02-12 21:18:45 +01:00
kakwa	0cf5483785	add warning in documentation for log level 'debug'	2019-02-10 18:19:55 +01:00
kakwa	df2746b996	version bump + changelog	2019-02-10 18:15:07 +01:00
kakwa	e6bcf9d97d	adding the possibility to log to stdout	2019-02-10 18:12:45 +01:00
kakwa	57bcaaed66	changelog and version bump	2019-02-09 20:49:23 +01:00
kakwa	b68214022c	fix error handling when adding user that already exists	2019-02-09 20:47:34 +01:00
kakwa	932e7a8b40	adding (mostly) working configuration example	2019-02-09 20:24:32 +01:00
kakwa	abf1454278	changelog+version bump	2019-02-09 20:21:29 +01:00
kakwa	0793361d90	switch to "stable" in setup.py troves	2019-02-09 20:19:57 +01:00
kakwa	f824790849	another exception	2019-02-09 19:42:48 +01:00
kakwa	7390c931b9	another exception	2019-02-09 19:37:52 +01:00
kakwa	4a8aa1c655	another exception	2019-02-09 19:32:50 +01:00
kakwa	e50df5dde3	wider exception for <input> attribute "type" has invalid value	2019-02-09 19:26:31 +01:00
kakwa	fba2d32b44	another exception for todylib	2019-02-09 19:21:41 +01:00
kakwa	7a8468f8b1	adding another ignore	2019-02-09 19:15:31 +01:00
kakwa	9d0d321e9b	another ignore for tidylib	2019-02-09 19:09:32 +01:00
kakwa	c5536bdc56	adding a fffew other exception in tidylib	2019-02-09 19:04:40 +01:00
kakwa	abfce4803a	fix typo	2019-02-09 18:57:50 +01:00
kakwa	046afbbe29	html_tidy cleanup	2019-02-09 18:54:35 +01:00
kakwa	98fca30fba	ignoring another nav error	2019-02-09 18:44:51 +01:00
kakwa	f13961790f	adding exception for <nav> tags in html validation	2019-02-09 18:40:48 +01:00
kakwa	a56c491ee1	cleanup in html template + tidylib * few small cleanup in html template (avoid empty tbody, put id between quotes) * switch to tidylib to validate the html instead of the previous hack calling an external service (https://html5.validator.nu/) * remove the previous validator script * add exception for tidylib on empty <span> (these are required by bootstrap)	2019-02-09 18:31:37 +01:00
kakwa	02357d886a	remove debug print	2019-02-09 18:18:58 +01:00
kakwa	263e6be547	fix html validator test for python 3	2019-02-09 17:40:43 +01:00
kakwa	05aace0e9d	force the groups in flatten roles to be sorted * sorting the groups helps debuggability and also permits testing that doesn't rely on python ordering (which is different between 2 and 3).	2019-02-09 17:36:01 +01:00
kakwa	baa3430e63	fix test and exception handling in code With python 2 it was possible to do exception[0][...] to recover details about an exception. It's no longer authorized with python 3. Now, we must do something like exception.args or exception.urls. fortunately this syntax also works with python 2. So we use it for both.	2019-02-09 17:12:39 +01:00
kakwa	90ff69586b	remove deprecation warning for html escape in python 2, (html) escape is part of the cgi module in python 3, it's part of the html module we now do a conditional import depending on the version, and name the function html_escape.	2019-02-09 16:29:16 +01:00
kakwa	79983c078f	fix behavior of get_attributes() * make sure it returns an ordered list in both python 2 and python 3	2019-02-09 16:22:42 +01:00
kakwa	10747cff93	add some python 3 support in the LDAP and AD backends python-ldap talks in bytes, as the rest of ldapcherry talks in unicode utf-8: * everything passed to python-ldap must be converted to bytes * everything coming from python-ldap must be converted to unicode The previous statement was true for python-ldap < version 3.X. With versions > 3.0.0 and python 3, it gets tricky, some parts of python-ldap takes string, specially the filters/escaper. so we have now: _byte_p2 (unicode -> bytes conversion for python 2) _byte_p3 (unicode -> bytes conversion for python 3) *_byte_p23 (unicode -> bytes conversion for python AND 3)	2019-02-09 16:08:18 +01:00
kakwa	979d4eeda8	disable ppolicy in samba test deployment	2019-02-09 15:42:48 +01:00
kakwa	fb6b0a5d31	limit cherrypy to < 18.0.0 in setup.py cherrypy dropped support for python2 with 18.0.0, 17.X is the last version usable with python 2.	2019-02-09 12:12:24 +01:00
kakwa	bbfe96d4f7	pep8	2019-02-09 12:05:09 +01:00
kakwa	b9437abefb	* support for python-ldap 2 and 3 * python-ldap 3 is slightly different than 2 on how it handles modify the modified attributes used to be transmitted as a dict, now it should be transmitted as a list of dict)	2019-02-09 11:58:09 +01:00
kakwa	60d57d8530	changelog	2019-02-08 20:47:15 +01:00
kakwa	8c0bf94904	better log+fix in conf checking + fix in ppolicy handler * log where the backend is declared (role or attribute) when inconsistency with main .ini file * fix check of configuration, only role file was checked 2 times instead on checking role one time and attribute one time * <dict>.keys() seems to have a different behavior between 2 (return "list") and 3 (return "dict_keys"), casting to "list" to avoid that.	2019-02-08 20:38:29 +01:00
kakwa	42759f1cc4	pep8	2019-02-08 20:38:03 +01:00
kakwa	18fdeb483e	better handling of the str/byte mess for python3 * add dedicated methods for python 3 in handling of bytearrays/strings * using them to compare attributes checks in AD backend	2019-02-08 20:33:58 +01:00
kakwa	12c511b537	switch to explicit bytearray for checking missing params	2019-02-08 00:11:01 +01:00
kakwa	d25ceef2d3	trying to fix samba/AD setup	2019-02-07 23:46:10 +01:00
kakwa	8b48a1f024	cleanup in travis file	2019-02-07 22:59:57 +01:00
kakwa	7430af5ffc	adding another samba package in test env	2019-02-07 22:44:51 +01:00
kakwa	bc0f3aceb5	adding another dependency for the samba/ad test	2019-02-07 22:42:30 +01:00
kakwa	9989f97091	remove python3 test env for el7 and stretch (no python3-ldap lib anyway)	2019-02-07 22:38:30 +01:00
kakwa	fc98b1bd70	fixing the test env deploy script + small fix in unit tests	2019-02-07 22:34:47 +01:00
kakwa	ab9cd664ec	fix pip install	2019-02-07 22:12:49 +01:00
kakwa	13bfbdcbbc	add requirements files for simulating RHEL 7 and Debian 9	2019-02-07 22:07:28 +01:00
kakwa	70140f966a	pep 8	2019-02-07 21:09:42 +01:00
kakwa	8bd4afb235	remove scripts from pycodestyle	2019-02-07 21:07:36 +01:00
kakwa	2a2864a306	porting the tests over to python3	2019-02-07 20:55:50 +01:00
kakwa	c3feafdb2c	pep8	2019-02-07 20:48:06 +01:00
kakwa	86fb6c1dd2	adding an update as the first step of the deploy script	2019-02-07 20:44:19 +01:00
kakwa	9f6af580cd	remove env that doesn't exist	2019-02-07 20:41:22 +01:00
kakwa	5bdcc5522a	switch to xenial in travis configuration	2019-02-07 20:40:22 +01:00
kakwa	c81429a870	few tweaks for python3 support * switch from script to entry_points in setup.py * move the cli script in ldapcherry (to be used as a module) * put the __main__ code in a dedicated function constituting the entry point * add a few python3 environments in travis file	2019-02-07 20:34:49 +01:00
kakwa	3d6e24eb73	pep8	2019-02-07 20:16:39 +01:00
kakwa	be598b0129	slightly cleaner testenv deploy script	2019-02-06 23:55:03 +01:00
kakwa	ccc252965d	fix another __import__	2019-02-06 23:04:23 +01:00
kakwa	3beedc8d4d	add an ignore on the local dev conf file	2019-02-06 23:03:55 +01:00
kakwa	74dc6c5894	various changes to support python3 * changes in urllib imports since quote_plus in urllib with python 2 and in urllib.parse in python 3 * changes in imports for Sets since set is a native type in python 3 and doesn't requires an import * fix in __import__, '-1' level for module path discovery is not supported anymore, switching to 0 (absolute import only).	2019-02-06 22:32:40 +01:00
kakwa	69526610f3	add a small script to generate a local dev config	2019-02-06 22:30:59 +01:00
kakwa	921a0820f4	switch to using lists in templates Sets are not available in mako templates when using python3. Reverting to using lists with 'if not in' checks to avoid duplication.	2019-02-06 22:26:46 +01:00
kakwa	2df56d2de2	fix template over-escaping + python 3 support The templates were html escaping the generated js code for the autofill and the role management. This was breaking these features. It's okay to not escape these as they are coming from a trusted source (configuration file). Also make the templates python3 compatible (not need to import Set in python 3)	2019-02-06 21:38:11 +01:00
Carpentier Pierre-Francois	5b0c72a572	Merge pull request #17 from jthiltges/escfix Escape form values with markupsafe	2019-01-03 23:39:53 +01:00
John Thiltges	c6cce54d5f	Escape form values with markupsafe - Use markupsafe to format escaped HTML fragments - Correct the formatting problems introduced with the XSS fixes	2019-01-03 13:12:53 -06:00
Carpentier Pierre-Francois	1f79648d57	Update ChangeLog.rst	2019-01-02 23:59:03 +01:00
Carpentier Pierre-Francois	636400b75f	Merge pull request #16 from jthiltges/escape Protect against XSS vulnerabilities in URL redirection	2019-01-02 23:54:42 +01:00
John Thiltges	6f98076281	Protect against XSS vulnerabilities in URL redirection - Switch from base64 to URL encoding for the passing the URL, using the built-in Mako filtering - Apply HTML filtering to Mako output by default - Disable HTML filtering for nested templates in adduser, modify, and selfmodify	2019-01-02 14:31:10 -06:00
Carpentier Pierre-Francois	1ed654c91b	Update README.rst	2018-02-07 19:54:23 +01:00
Carpentier Pierre-Francois	c329e53811	Update README.rst	2018-02-07 19:52:29 +01:00
Carpentier Pierre-Francois	05e3a0d665	Update README.rst	2017-10-26 10:08:16 +02:00
kakwa	4bd6314b3b	remove useless tests	2017-06-12 19:50:42 +02:00
kakwa	c5dae7039a	remove duplicated import in docs conf.py	2017-06-12 19:47:43 +02:00
kakwa	ca1f78173f	better documenation	2017-06-09 23:40:23 +02:00
kakwa	9ed6007b02	including fastcgi configuration example in the documentation	2017-06-09 23:25:58 +02:00
kakwa	4d696a29ef	adding example for unix socket in defautl conf	2017-06-09 23:24:20 +02:00
kakwa	45d64120ae	adding an nginx configuration exmaple for fastcgi	2017-06-09 23:09:11 +02:00
kakwa	00a4d22dd9	remove pip install method	2017-04-06 21:53:58 +02:00
kakwa	32c513f96e	change install method (pip install just doesn't work)	2017-04-06 21:37:02 +02:00
kakwa	7019cc2348	fix setup.py	2017-04-06 20:58:20 +02:00
kakwa	a404cf0b39	add auto message for tagging script	2017-04-06 20:57:42 +02:00
kakwa	9649803dd6	changelog	2017-04-06 20:52:55 +02:00
kakwa	eecccac106	fix import of version in docs/conf.py and setup.py	2017-04-06 20:46:58 +02:00
kakwa	f357adcd9a	put version in standalone file this way, it avoids error due to missing imports	2017-04-06 20:34:32 +02:00
kakwa	e7998ced78	adding a simple tagging script	2017-04-06 20:28:44 +02:00
kakwa	8270988ed4	changelog + version bump + factorize version	2017-04-06 20:21:31 +02:00
kakwa	2e2453f309	fix camelcase	2017-04-06 01:26:54 +02:00
kakwa	bbb13454bf	more warning removal	2017-04-06 01:21:57 +02:00
kakwa	3378822d2e	fix some warnings	2017-04-06 01:20:51 +02:00
kakwa	6e526b6f15	hack to have a cleaner resize	2017-04-06 00:32:24 +02:00