Merge branch 'install_doc' into 'develop'

Restructure/improve docs

Closes #1345

See merge request funkwhale/funkwhale!1314

CHANGELOG

@@ -882,8 +882,8 @@ Tags are used in various places to enhance user experience:
 - The custom radio builder now supports using tags
 - Subsonic apps that support genres - such as DSub or Ultrasonic - should display this information as well
 
-If you are a pod admin and want to extract tags from already uploaded content, you run `this snippet <https://dev.funkwhale.audio/funkwhale/funkwhale/snippets/43>`_
+If you are a pod admin and want to extract tags from already uploaded content, you run `this snippet <https://dev.funkwhale.audio/funkwhale/funkwhale/snippets/43>`__
-and `this snippet <https://dev.funkwhale.audio/funkwhale/funkwhale/snippets/44>`_ in a ``python manage.py shell``.
+and `this snippet <https://dev.funkwhale.audio/funkwhale/funkwhale/snippets/44>`__ in a ``python manage.py shell``.
 
 Content and account reports
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -899,8 +899,8 @@ Federation of the reports will be supported in a future release.
 
 For more information about this feature, please check out our documentation:
 
--  `User documentation <https://docs.funkwhale.audio/moderator/reports.html>`_
+-  `User documentation <https://docs.funkwhale.audio/moderator/reports.html>`__
--  `Moderator documentation <https://docs.funkwhale.audio/users/reports.html>`_
+-  `Moderator documentation <https://docs.funkwhale.audio/users/reports.html>`__
 
 Account deletion
 ^^^^^^^^^^^^^^^^
@@ -913,7 +913,7 @@ to other known servers on the federation.
 
 For more information about this feature, please check out our documentation:
 
--  `User documentation <https://docs.funkwhale.audio/users/account.html>`_
+-  `User documentation <https://docs.funkwhale.audio/users/account.html>`__
 
 Landing and about page redesign [Manual action suggested]
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -977,7 +977,7 @@ Both messages will appear for the first time 15 days after signup, in the notifi
 
 
 Replaced Daphne by Gunicorn/Uvicorn [manual action required, non-docker only]
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To improve the performance, stability and reliability of Funkwhale's web processes,
 we now recommend using Gunicorn and Uvicorn instead of Daphne. This combination unlock new use cases such as:
@@ -1295,7 +1295,7 @@ special permissions, such as modifying instance settings or moderation (but this
 enabled in a future release).
 
 If you want to start building an app on top of Funkwhale's API, please check-out
-`https://docs.funkwhale.audio/api.html`_ and `https://docs.funkwhale.audio/developers/authentication.html`_.
+https://docs.funkwhale.audio/api.html and https://docs.funkwhale.audio/developers/authentication.html.
 
 Better error handling and display during import
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -1313,7 +1313,7 @@ Storing all media files on the Funkwhale server itself may not be possible or de
 in all scenarios. You can now configure Funkwhale to store those files in a S3
 bucket instead.
 
-Check-out `https://docs.funkwhale.audio/admin/external-storages.html`_ if you want to use
+Check-out https://docs.funkwhale.audio/admin/external-storages.html if you want to use
 this feature.
 
 Prune library command
@@ -1324,7 +1324,7 @@ metadata even if no associated files exist.
 
 To help with that, we now offer a ``prune_library`` management command you can run
 to purge your database from obsolete entries. `Please refer to our documentation
-for usage instructions <https://docs.funkwhale.audio/admin/commands.html#pruning-library>`_.
+for usage instructions <https://docs.funkwhale.audio/admin/commands.html#pruning-library>`__.
 
 Check in-place files command
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -1335,7 +1335,7 @@ files in the database, which results in unplayable tracks.
 
 To help with that, we now offer a ``check_inplace_files`` management command you can run
 to purge your database from obsolete files. `Please refer to our documentation
-for usage instructions <https://docs.funkwhale.audio/admin/commands.html#remove-obsolete-files-from-database>`_.
+for usage instructions <https://docs.funkwhale.audio/admin/commands.html#remove-obsolete-files-from-database>`__.
 
 
 Features:
@@ -2994,7 +2994,7 @@ Starting from this release, we now offer a dedicated interface directly
 in the front-end. You can view and edit all your instance settings from here,
 assuming you have the required permissions.
 
-This interface is available at ``/manage/settings` and via link in the sidebar.
+This interface is available at ``/manage/settings`` and via link in the sidebar.
 
 
 Storage of bitrate, size and length in database
@@ -3626,7 +3626,7 @@ Features:
 - Switched to django-channels and daphne for serving HTTP and websocket (#34)
 
 Upgrades notes
-**************
+^^^^^^^^^^^^^^
 
 This version contains breaking changes in the way funkwhale is deployed,
 please read the notes carefully.

changes/changelog.d/1314.doc

@@ -0,0 +1 @@
+Added server uninstallation documentation (\!1314)

changes/changelog.d/1345.doc

@@ -0,0 +1 @@
+Fixed broken backup documentation (#1345)

changes/changelog.d/refactored.doc

@@ -0,0 +1 @@
+Refactored installation documentation and other small documentation adjustments (\!1314)

docs/admin/backup.rst

@@ -14,8 +14,8 @@ If you've followed the setup instructions in :doc:`../installation/docker`, here
 Multi-container installation
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Backup the db
+Backup the database
-^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^
 
 On docker setups, you have to ``pg_dumpall`` in container ``funkwhale_postgres_1``:
 
@@ -47,8 +47,8 @@ On docker setups, the configuration file is located at the root level:
 Non-docker setup
 ----------------
 
-Backup the db
+Backup the database
-^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^
 
 On non-docker setups, you have to ``pg_dump`` as user ``postgres``:
 

docs/admin/configuration.rst

@@ -2,8 +2,9 @@ Instance configuration
 ======================
 
 General configuration is achieved using two type of settings:
-:ref:`environment variables <environment-variables>` and
+
-:ref:`instance settings <instance-settings>`.
+- :ref:`environment variables <environment-variables>` and
+- :ref:`instance settings <instance-settings>`.
 
 .. _environment-variables:
 
@@ -11,8 +12,7 @@ Environment variables
 ---------------------
 
 Those are located in your ``.env`` file, which you should have created
-during installation. A full list of available variables can be seen
+during installation. A full list of available variables is given :ref:`below <environment-variables>`.
-:ref:`below <environment-variables>`.
 
 Options from this file are heavily commented, and usually target lower level
 and technical aspects of your instance, such as database credentials.
@@ -20,7 +20,9 @@ and technical aspects of your instance, such as database credentials.
 .. note::
 
     You should restart all Funkwhale processes when you change the values
-    on environment variables.
+    on environment variables::
+        
+        sudo systemctl restart funkwhale.target
 
 
 .. note::
@@ -28,10 +30,11 @@ and technical aspects of your instance, such as database credentials.
     Some characters are unsafe to use in configuration variables that are URLs,
     such as the user and password in the database and SMTP sections.
     If those variables contain such characters, they must be urlencoded, for
-    instance using the following command:
+    instance using the following command::
-    ``python3 -c 'import urllib.parse; print(urllib.parse.quote_plus("p@ssword"))``
 
-    cf. https://github.com/joke2k/django-environ#using-unsafe-characters-in-urls
+        python3 -c 'import urllib.parse; print(urllib.parse.quote_plus("p@ssword"))
+
+    See as well https://github.com/joke2k/django-environ#using-unsafe-characters-in-urls
 
 .. _instance-settings:
 
@@ -186,21 +189,20 @@ User permissions
 
 Funkwhale's permission model works as follows:
 
-- Anonymous users cannot do anything unless configured specifically
+- Anonymous users cannot do anything unless configured specifically;
-- Logged-in users can use the application, but cannot do things that affect
+- Logged-in users can use the application, but cannot do things that affect the whole instance;
-  the whole instance
+- Superusers can do anything.
-- Superusers can do anything
 
 To make things more granular and allow some delegation of responsibility,
 superusers can grant specific permissions to specific users. Available
 permissions are:
 
 - **Manage instance-level settings**: users with this permission can edit instance
-  settings as described in :ref:`instance-settings`
+  settings as described in :ref:`instance-settings`;
 - **Manage library**: users with this permission can import new music in the
-  instance
+  instance;
 - **Manage library federation**: users with this permission can ask to federate with
-  other instances, and accept/deny federation requests from other instances
+  other instances, and accept/deny federation requests from other instances.
 
 There is no dedicated interface to manage users permissions, but superusers
 can login on the :doc:`Django's admin <django>` at ``/api/admin/`` and grant permissions
@@ -216,7 +218,7 @@ on ``https://yourinstanceurl/settings.json``.
 On typical deployments, this url returns a 404 error, which is simply ignored.
 
 Set-up
-------
+^^^^^^
 
 First, create the settings file:
 
@@ -244,11 +246,11 @@ If you are using nginx, add the following snippet to your vhost configuration::
         alias /srv/funkwhale/custom/settings.json;
     }
 
-On apache, add the following to your vhost configuration::
+On Apache, add the following to your vhost configuration::
 
     Alias /settings.json /srv/funkwhale/custom/settings.json
 
-Then reload your reverse proxy.
+Then, reload your reverse proxy.
 
 At this point, visiting ``https://yourinstanceurl/settings.json`` should serve the content
 of the settings.json file.
@@ -285,8 +287,8 @@ Theming
 
 To theme your Funkwhale instance, you need:
 
-1. A CSS file for your theme, that can be loaded by the front-end
+1. A CSS file for your theme, that can be loaded by the front-end;
-2. To update the value of ``additionalStylesheets`` in your settings.json file to point to your CSS file URL
+2. To update the value of ``additionalStylesheets`` in your settings.json file to point to your CSS file URL.
 
 .. code-block:: shell
 
@@ -311,7 +313,7 @@ On nginx, add the following snippet to your vhost config::
         alias /srv/funkwhale/custom;
     }
 
-On apache, use the following::
+On Apache, use the following::
 
     Alias /custom /srv/funkwhale/custom
 

docs/admin/importing-music.rst

@@ -93,8 +93,8 @@ following behaviour during import:
 Because those files are not managed by Funkwhale, we offer additional
 configuration options to ensure the webserver can serve them properly:
 
-- :ref:`setting-MUSIC_DIRECTORY_PATH`
+- :data:`MUSIC_DIRECTORY_PATH <config.settings.common.MUSIC_DIRECTORY_PATH>`
-- :ref:`setting-MUSIC_DIRECTORY_SERVE_PATH`
+- :data:`MUSIC_DIRECTORY_SERVING_PATH <config.settings.common.MUSIC_DIRECTORY_SERVE_PATH>`
 
 We recommend you symlink all your music directories into ``/srv/funkwhale/data/music``
 and run the `import_files` command from that directory. This will make it possible

docs/admin/index.rst

@@ -15,7 +15,11 @@ Setup Guides
    configuration
    importing-music
    external-storages
+   optimization
+   backup
    migration
+   uninstall
+   ldap
 
 Administration
 --------------
@@ -29,10 +33,11 @@ Administration
    upgrading
    mrf
 
-Troubleshooting Issues
+Troubleshooting
-----------------------
+---------------
 
 .. toctree::
    :maxdepth: 2
 
    troubleshooting
+   debugging

docs/admin/ldap.rst

@@ -7,9 +7,7 @@ Funkwhale supports LDAP through the Django LDAP authentication module and by set
 
 .. warning::
 
-    Note the following restrictions when using LDAP:
+    Note that LDAP-based users cannot change their password inside the app.
-
-        * LDAP-based users cannot change passwords inside the app.
 
 Dependencies
 ------------
@@ -21,23 +19,25 @@ Environment variables
 
 LDAP authentication is configured entirely through the environment variables. The following options enable the LDAP features:
 
-Basic features:
+Basic features
+^^^^^^^^^^^^^^
 
-* ``LDAP_ENABLED``: Set to ``True`` to enable LDAP support. Default: ``False``.
+- ``LDAP_ENABLED``: Set to ``True`` to enable LDAP support. Default: ``False``.
-* ``LDAP_SERVER_URI``: LDAP URI to the authentication server, e.g. ``ldap://my.host:389``.
+- ``LDAP_SERVER_URI``: LDAP URI to the authentication server, e.g. ``ldap://my.host:389``.
-* ``LDAP_BIND_DN``: LDAP user DN to bind as to perform searches.
+- ``LDAP_BIND_DN``: LDAP user DN to bind as to perform searches.
-* ``LDAP_BIND_PASSWORD``: LDAP user password for bind DN.
+- ``LDAP_BIND_PASSWORD``: LDAP user password for bind DN.
-* ``LDAP_SEARCH_FILTER``: The LDAP user filter, using ``{0}`` as the username placeholder, e.g. ``(|(cn={0})(mail={0}))``; uses standard LDAP search syntax. Default: ``(uid={0})``.
+- ``LDAP_SEARCH_FILTER``: The LDAP user filter, using ``{0}`` as the username placeholder, e.g. ``(|(cn={0})(mail={0}))``; uses standard LDAP search syntax. Default: ``(uid={0})``.
-* ``LDAP_START_TLS``: Set to ``True`` to enable LDAP StartTLS support. Default: ``False``.
+- ``LDAP_START_TLS``: Set to ``True`` to enable LDAP StartTLS support. Default: ``False``.
-* ``LDAP_ROOT_DN``: The LDAP search root DN, e.g. ``dc=my,dc=domain,dc=com``; supports multiple entries in a space-delimited list, e.g. ``dc=users,dc=domain,dc=com dc=admins,dc=domain,dc=com``.
+- ``LDAP_ROOT_DN``: The LDAP search root DN, e.g. ``dc=my,dc=domain,dc=com``; supports multiple entries in a space-delimited list, e.g. ``dc=users,dc=domain,dc=com dc=admins,dc=domain,dc=com``.
-* ``LDAP_USER_ATTR_MAP``: A mapping of Django user attributes to LDAP values, e.g. ``first_name:givenName, last_name:sn, username:cn, email:mail``. Default: ``first_name:givenName, last_name:sn, username:cn, email:mail``.
+- ``LDAP_USER_ATTR_MAP``: A mapping of Django user attributes to LDAP values, e.g. ``first_name:givenName, last_name:sn, username:cn, email:mail``. Default: ``first_name:givenName, last_name:sn, username:cn, email:mail``.
-* ``AUTH_LDAP_BIND_AS_AUTHENTICATING_USER``: Controls whether direct binding is used. Default: ``False``.
+- ``AUTH_LDAP_BIND_AS_AUTHENTICATING_USER``: Controls whether direct binding is used. Default: ``False``.
 
-Group features:
+Group features
+^^^^^^^^^^^^^^
 
-For details on these options, see `the Django documentation <https://django-auth-ldap.readthedocs.io/en/latest/groups.html>`_. Group configuration is disabled unless an ``LDAP_GROUP_DN`` is set. This is an advanced LDAP feature and most users should not need to configure these settings.
+For details on these options, see the `Django documentation <https://django-auth-ldap.readthedocs.io/en/latest/groups.html>`_. Group configuration is disabled unless an ``LDAP_GROUP_DN`` is set. This is an advanced LDAP feature and most users should not need to configure these settings.
 
-* ``LDAP_GROUP_DN``: The LDAP group search root DN, e.g. ``ou=groups,dc=domain,dc=com``.
+- ``LDAP_GROUP_DN``: The LDAP group search root DN, e.g. ``ou=groups,dc=domain,dc=com``.
-* ``LDAP_GROUP_FILTER``: The LDAP group filter, e.g. ``(objectClass=groupOfNames)``.
+- ``LDAP_GROUP_FILTER``: The LDAP group filter, e.g. ``(objectClass=groupOfNames)``.
-* ``LDAP_REQUIRE_GROUP``: A group users must be a part of to authenticate, e.g. ``cn=enabled,ou=groups,dc=domain,dc=com``.
+- ``LDAP_REQUIRE_GROUP``: A group users must be a part of to authenticate, e.g. ``cn=enabled,ou=groups,dc=domain,dc=com``.
-* ``LDAP_DENY_GROUP``: A group users must not be a part of to authenticate, e.g. ``cn=disabled,ou=groups,dc=domain,dc=com``.
+- ``LDAP_DENY_GROUP``: A group users must not be a part of to authenticate, e.g. ``cn=disabled,ou=groups,dc=domain,dc=com``.

docs/admin/migration.rst

@@ -6,6 +6,8 @@ existing Funkwhale setup to a new server. This can be helpful
 if you need to boost resources or if you wish to use a different
 hosting platform.
 
+In this guide, the existing Funkwhale setup is called the origin server, and the new setup the destination server.
+
 Requirements
 ------------
 
@@ -18,67 +20,83 @@ following:
 Non-Docker
 ----------
 
-- On the target server, run through the :doc:`installation steps<../installation/debian>` but skip the Database setup steps
+On the destination server, run through the :doc:`installation steps<../installation/debian>` with the exception of the following points:
-- Stop all funkwhale related services on the destination server
+
-- On the original server, create a database backup
+- Do not enable the extensions ``unaccent`` and ``citext`` when setting up the database;
+- Do not initialize the database by applying the migrate command;
+- Do not create an admin account.
+
+Stop all funkwhale related services on the destination server:
+
+.. code-block:: shell
+
+    sudo systemctl stop funkwhale.target
+
+On the origin server, create a database backup:
 
 .. code-block:: shell
 
     sudo -u funkwhale pg_dump -d funkwhale > "db.dump"
 
-- On the destination server, use rsync to copy the contents of `/srv/funwkhale/data/media/music` and `/srv/funkwhale/data/music` from the original server
+On the destination server, use rsync to fetch the contents of ``/srv/funwkhale/data/media/music/`` and ``/srv/funkwhale/data/media/`` from the origin server, as well as the database dump and the ``.env`` file:
 
 .. code-block:: shell
 
-    rsync -a <your username>@<original server IP/hostname>:/srv/funkwhale/data/media/ /srv/funkwhale/data/media/
+    origin = <origin server IP/hostname>
-    rsync -a <your username>@<original server IP/hostname>:/srv/funkwhale/data/music/ /srv/funkwhale/data/music/
+    username = <your username>
 
-- Copy your .env file and database backup from your original server
+    rsync -a $username@$origin:/srv/funkwhale/data/media/ /srv/funkwhale/data/media/
+    rsync -a $username@$origin:/srv/funkwhale/data/music/ /srv/funkwhale/data/music/
+
+    rsync -a $username@$origin:/srv/funkwhale/config/.env /srv/funkwhale/config/
+    rsync -a $username@$origin:/srv/funkwhale/db.dump /srv/funkwhale/
+
+On the destination server, restore the database dump:
 
 .. code-block:: shell
 
-    rsync -a <your username>@<original server IP/hostname>:/srv/funkwhale/config/.env /srv/funkwhale/config/
+    sudo psql -d funkwhale db.dump
-    rsync -a <your username>@<original server IP/hostname>:/srv/funkwhale/db.dump /srv/funkwhale/
 
-- Restore the database dump
+Once the database has been restored, follow the database migration steps from the non-docker installation guide to complete the installation on the destination server.
+
+Ensure that all DNS changes have been made and start the services:
 
 .. code-block:: shell
 
-    sudo -u funkwhale pg_restore -d funkwhale db.dump
+    sudo systemctl start funkwhale.target
-
-- Once the database has been restored, follow the database migration steps from the guide to complete the installation
-- Ensure that all DNS changes have been made and start the services
 
 Docker
 ------
 
-- On the target server, run through the :doc:`installation steps<../installation/docker>` but skip the `docker-compose run --rm api python manage.py migrate` step
+On the destination server, run through the :doc:`installation steps<../installation/docker>` but skip the ``docker-compose run --rm api python manage.py migrate`` step.
-- Stop all funkwhale related containers on the destination server
+
-- On the original server, create a database backup
+Stop all funkwhale related containers on the destination server.
+
+On the origin server, create a database backup:
 
 .. code-block:: shell
 
     docker exec -t funkwhale_postgres_1 pg_dumpall -c -U postgres > "db.dump"
 
-- On the destination server, use rsync to copy the contents of `/srv/funwkhale/data/media/music` and `/srv/funkwhale/data/music` from the original server
+On the destination server, use rsync to fetch the contents of ``/srv/funwkhale/data/media/music`` and ``/srv/funkwhale/data/media`` from the origin server, as well as the database dump nd the ``.env`` file:
 
 .. code-block:: shell
 
-    rsync -a <your username>@<original server IP/hostname>:/srv/funkwhale/data/media/ /srv/funkwhale/data/media/
+    origin = <origin server IP/hostname>
-    rsync -a <your username>@<original server IP/hostname>:/srv/funkwhale/data/music/ /srv/funkwhale/data/music/
+    username = <your username>
 
-- Copy your .env file and database backup from your original server
+    rsync -a $username@$origin:/srv/funkwhale/data/media/ /srv/funkwhale/data/media/
+    rsync -a $username@$origin:/srv/funkwhale/data/music/ /srv/funkwhale/data/music/
 
-.. code-block:: shell
+    rsync -a $username@$origin:/srv/funkwhale/.env /srv/funkwhale/
+    rsync -a $username@$origin:/srv/funkwhale/db.dump /srv/funkwhale/
 
-    rsync -a <your username>@<original server IP/hostname>:/srv/funkwhale/.env /srv/funkwhale/
+Restore the database dump:
-    rsync -a <your username>@<original server IP/hostname>:/srv/funkwhale/db.dump /srv/funkwhale/
-
-- Restore the database dump
 
 .. code-block:: shell
 
     docker exec -i funkwhale_postgres_1 pg_restore -c -U postgres -d postgres < "db.dump"
 
-- Once the database has been restored, run the migrations
+Once the database has been restored, run the migrations following the docker installation guide. 
-- Ensure that all DNS changes have been made and start the services
+
+Ensure that all DNS changes have been made and start the services.

docs/admin/mrf.rst

@@ -13,8 +13,8 @@ of registered policies, in turn. Each policy can mutate the message, leave it as
 
 Some of our built-in moderation tools are actually implemented as a MRF policy, e.g:
 
-- Allow-list, when checking incoming messages (`code <https://dev.funkwhale.audio/funkwhale/funkwhale/blob/develop/api/funkwhale_api/moderation/mrf_policies.py>`_)
+- Allow-list, when checking incoming messages (`code <https://dev.funkwhale.audio/funkwhale/funkwhale/blob/develop/api/funkwhale_api/moderation/mrf_policies.py>`__)
-- Domain and user blocking, when checking incoming messages (`code <https://dev.funkwhale.audio/funkwhale/funkwhale/blob/develop/api/funkwhale_api/federation/mrf_policies.py>`_)
+- Domain and user blocking, when checking incoming messages (`code <https://dev.funkwhale.audio/funkwhale/funkwhale/blob/develop/api/funkwhale_api/federation/mrf_policies.py>`__)
 
 .. note::
 

docs/admin/troubleshooting.rst

@@ -50,12 +50,12 @@ Diagnostic tools:
 Common problems
 ***************
 
-Instance work properly, but audio files are not served (404 error)
+Instance works properly, but audio files are not served (404 error)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 - If you're using docker, ensure the ``MEDIA_ROOT`` variable is commented in your env file
 - Ensure the ``_protected/media`` block points toward the path where media files are stored (``/srv/funkwhale/data/media``, by default)
-- If you're using in-place import, ensure :ref:`setting-MUSIC_DIRECTORY_PATH`, :ref:`setting-MUSIC_DIRECTORY_SERVE_PATH` and :ref:`setting-REVERSE_PROXY_TYPE` are configured properly, and that the files are readable by the webserver
+- If you're using in-place import, ensure :data:`MUSIC_DIRECTORY_PATH <config.settings.common.MUSIC_DIRECTORY_PATH>`, :data:`MUSIC_DIRECTORY_SERVE_PATH <config.settings.common.MUSIC_DIRECTORY_SERVE_PATH>` and :data:`REVERSE_PROXY_TYPE <config.settings.common.REVERSE_PROXY_TYPE>` are configured properly, and that the files are readable by the webserver
 
 Weakref error when running ``python manage.py <command>``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

docs/admin/uninstall.rst

@@ -0,0 +1,89 @@
+Uninstall Funkwhale
+===================
+
+The following instructions helps you remove Funkwhale from your server, for instance after migrating to another server, or if you do not want to use Funkwhale anymore.
+
+.. warning::
+
+    The following instructions cannot be undone and might result in loss of data. If necessary, please make a backup of your server following the :doc:`backup instructions<backup>`.
+
+    Especially, it must be noted that:
+
+    - Remote content hosted on an S3 or S3-compatible server will not be removed.
+    - In place imports will not be removed, provided they are located outside ``/srv/funkwhale/`` 
+
+.. note::
+
+    These instructions apply only for the manual installation on Debian or Arch Linux. It matches the default setup.
+
+First, stop the all funkwhale related services:
+
+.. code-block:: shell
+
+    sudo systemctl stop funkwhale.target
+
+Remove the reverse proxy configuration data and reload the reverse proxy.
+
+If you are using nginx:
+
+.. code-block:: shell
+
+    sudo rm /etc/nginx/sites-enabled/funkwhale.conf    
+    sudo rm /etc/nginx/sites-available/funkwhale.conf
+    sudo rm /etc/nginx/funkwhale_proxy.conf
+
+    sudo systemctl reload nginx
+
+If you are using Apache2:
+
+.. code-block:: shell
+
+    sudo rm /etc/apache2/sites-enabled/funkwhale.conf    
+    sudo rm /etc/apache2/sites-available/funkwhale.conf
+
+    sudo service apache2 restart
+
+Remove the systemd services:
+
+.. code-block:: shell
+
+    sudo systemctl disable funkwhale-server
+    sudo systemctl disable funkwhale-worker
+    sudo systemctl disable funkwhale-beat
+
+    sudo rm /etc/systemd/system/funkwhale-server.service
+    sudo rm /etc/systemd/system/funkwhale-worker.service
+    sudo rm /etc/systemd/system/funkwhale-beat.service
+    sudo rm /etc/systemd/system/funkwhale.target
+
+    sudo systemctl daemon-reload
+    sudo systemctl reset-failed
+
+Then, remove the database:
+
+.. code-block:: shell
+
+    sudo -u postgres psql -c 'DROP DATABASE funkwhale;'
+    sudo -u postgres psql -c 'DROP USER funkwhale;'
+
+Finally, remove the user ``funkwhale`` and all funkwhale related data, including the server and the data:
+
+.. code-block:: shell
+
+    sudo userdel -r funkwhale
+
+.. warning::
+
+    The last command will remove ``/srv/funkwhale/``. On the default setup, this directory contains all user data. Please proceed cautiously!
+
+    However, it must be noted that:
+
+    - Remote content hosted on an S3 or S3-compatible server will not be removed.
+    - In place imports will not be removed, provided they are not located in the directory ``/srv/funkwhale/``
+
+.. note::
+
+    If relevant, you might also want to:
+
+    - remove the SSL certificates;
+    - remove the corresponding DNS entries.

docs/backup.rst

@@ -1,79 +0,0 @@
-Backup your Funkwhale instance
-==============================
-
-.. note::
-
-    Before upgrading your instance, we strongly advise you to make at least a database backup. Ideally, you should make a full backup, including the database and the media files.
-
-
-Docker setup
-------------
-
-If you've followed the setup instructions in :doc:`../installation/docker`, here is the backup path:
-
-Multi-container installation
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Backup the db
-^^^^^^^^^^^^^
-
-On docker setups, you have to ``pg_dumpall`` in container ``funkwhale_postgres_1``:
-
-.. code-block:: shell
-
-   docker exec -t funkwhale_postgres_1 pg_dumpall -c -U postgres > dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql
-
-Backup the media files
-^^^^^^^^^^^^^^^^^^^^^^
-
-To backup docker data volumes, as the volumes are bound mounted to the host, the ``rsync`` way would go like this:
-
-.. code-block:: shell
-
-    rsync -avzhP /srv/funkwhale/data/media /path/to/your/backup/media
-    rsync -avzhP /srv/funkwhale/data/music /path/to/your/backup/music
-
-
-Backup the configuration files
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-On docker setups, the configuration file is located at the root level:
-
-.. code-block:: shell
-
-    rsync -avzhP /srv/funkwhale/.env /path/to/your/backup/.env
-
-
-Non-docker setup
-----------------
-
-Backup the db
-^^^^^^^^^^^^^
-
-On non-docker setups, you have to ``pg_dump`` as user ``postgres``:
-
-.. code-block:: shell
-
-   sudo -u postgres -H pg_dump funkwhale > /path/to/your/backup/dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql
-
-Backup the media files
-^^^^^^^^^^^^^^^^^^^^^^
-
-A simple way to backup your media files is to use ``rsync``:
-
-.. code-block:: shell
-
-    rsync -avzhP /srv/funkwhale/data/media /path/to/your/backup/media
-    rsync -avzhP /srv/funkwhale/data/music /path/to/your/backup/music
-
-Backup the configuration files
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: shell
-
-    rsync -avzhP /srv/funkwhale/config/.env /path/to/your/backup/.env
-
-.. note::
-   You may also want to backup your proxy configuration file.
-
-   For frequent backups, you may want to use deduplication and compression to keep the backup size low. In this case, a tool like ``borg`` will be more appropriate.

docs/cli/examples.rst

@@ -1,64 +0,0 @@
-Funkwhale CLI examples
-======================
-
-Uploading local files
----------------------
-
-**Goal**: create a library and upload all MP3 files from ``~/Music`` to it
-
-**Commands**::
-
-    funkwhale libraries create --name "My awesome library" --visibility me
-    # copy the returned UUID
-    funkwhale uploads create <UUID> ~/Music/**/*.mp3
-
-
-Favorite an entire album
-------------------------
-
-**Goal**: retrieve all the tracks from an album and add these to your favorites
-
-**Commands**::
-
-    # retrieve the album ID
-    funkwhale albums ls "The Slip"
-
-    # Copy the ID, then retrieve 100 pages of tracks from that album
-    # get only the IDs and pipe those to the favorite creation command
-    funkwhale tracks ls -f "album=<ID>" --ids --limit 100 \
-        | xargs funkwhale favorites tracks create
-
-
-Mirror an artist discography locally
-------------------------------------
-
-**Goal**: Download the discography of an artist locally, in the ``~/Music`` directory, in an ``Artist/Album/Track`` folder hierarchy
-
-**Commands**::
-
-    # retrieve the artist ID
-    funkwhale artists ls "Nine Inch Nails"
-
-    # Copy the ID, then retrieve 100 pages of tracks from that artist
-    # get only the IDs and pipe those to the download command
-    funkwhale tracks ls -f "artist=<ID>" --ids --limit 100 \
-        | xargs funkwhale tracks download \
-            -f mp3 -d ~/Music -t "{artist}/{album}/{title}.{extension}"
-
-
-Open a remote album in VLC
---------------------------
-
-**Goal**: Variation of the previous example, but instead of downloading an artist discography, we listen to an album in VLC
-
-**Commands**::
-
-    # retrieve the album ID
-    funkwhale albums ls "The Slip"
-
-    # Copy the ID, then retrieve 100 pages of tracks from that album
-    # get only the IDs, download the corresponding tracks and pipe the audio
-    # directly to VLC
-    funkwhale tracks ls -f "album=<ID>" --ids --limit 100 \
-        | xargs funkwhale tracks download \
-        | vlc -

docs/conf.py

@@ -64,8 +64,8 @@ master_doc = "index"
 # General information about the project.
 year = datetime.datetime.now().year
 project = "funkwhale"
-copyright = "{}, Eliot Berriot".format(year)
+copyright = "{}, The Funkwhale Collective".format(year)
-author = "Eliot Berriot"
+author = "The Funkwhale Collective"
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
@@ -107,7 +107,7 @@ html_theme = "sphinx_rtd_theme"
 # further.  For a list of options available for each theme, see the
 # documentation.
 #
-html_theme_options = {"gitlab_url": "https://dev.funkwhale.audio/funkwhale/funkwhale"}
+#html_theme_options = {}
 html_context = {
     "display_gitlab": True,
     "gitlab_host": "dev.funkwhale.audio",
@@ -115,13 +115,15 @@ html_context = {
     "gitlab_user": "funkwhale",
     "gitlab_version": "master",
     "conf_py_path": "/docs/",
+    "gitlab_url": "https://dev.funkwhale.audio/funkwhale/funkwhale",
 }
 html_logo = "logo.svg"
+html_favicon = "../front/public/favicon.png"
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ["_static"]
+# html_static_path = ["_static"]
 
 # -- Options for HTMLHelp output ------------------------------------------
 
@@ -190,6 +192,13 @@ redirect_files = [
     ("upgrading/index.html", "../admin/upgrading.html"),
     ("upgrading/0.17.html", "../admin/0.17.html"),
     ("users/django.html", "../admin/django.html"),
+    ("cli/index.html", "../users/cli.html"),
+    ("cli/examples.html", "../users/cli.html#examples"),
+    ("installation/ldap.html", "../admin/ldap.html"),
+    ("installation/optimization.html", "../admin/optimization.html"),
+    ("installation/external_dependencies.html", "debian.html"),
+    ("installation/systemd.html", "debian.html#systemd-unit-file"),
+    ("backup.html", "../admin/backup.html"),
 ]
 
 # Generate redirect template

docs/developers/index.rst

@@ -4,9 +4,6 @@ Developer documentation
 This documentation is targeted primarily at developers who want to understand
 how Funkwhale works and how to build apps that integrate with Funkwhale's ecosystem.
 
-Reference
----------
-
 .. toctree::
    :maxdepth: 2
 

docs/developers/plugins.rst

@@ -160,6 +160,6 @@ Filters reference
 
 .. autodata:: config.plugins.PLUGINS_DEPENDENCIES
 .. autodata:: config.plugins.PLUGINS_APPS
-.. autodata:: config.plugins.PLUGINSMIDDLEWARES_BEFORE_DEPENDENCIES
+.. autodata:: config.plugins.MIDDLEWARES_BEFORE
 .. autodata:: config.plugins.MIDDLEWARES_AFTER
 .. autodata:: config.plugins.URLS

docs/installation/debian.rst

@@ -3,15 +3,30 @@ Debian and Arch Linux installation
 
 .. note::
 
-    This guide targets Debian 9 (Stretch), which is the latest Debian, as well as Arch Linux.
+    This guide targets Debian 10 (Buster), which is the latest version available, as well as Arch Linux.
 
-External dependencies
+Cache setup (Redis)
----------------------
+-------------------
 
-The guides will focus on installing Funkwhale-specific components and
+Funkwhale requires a cache server:
-dependencies. However, Funkwhale requires a
+
-:doc:`few external dependencies <./external_dependencies>` for which
+- to make the whole system faster, by caching network requests or database queries;
-documentation is outside of this document scope.
+- to handle asynchronous tasks such as music import.
+
+On Debian-like distributions, a redis package is available, and you can
+install it:
+
+.. code-block:: shell
+
+    sudo apt-get install redis-server
+
+On Arch Linux and its derivatives:
+
+.. code-block:: shell
+
+    sudo pacman -S redis
+
+This should be enough to have your redis server set up.
 
 Install system dependencies
 ---------------------------
@@ -21,8 +36,9 @@ On Debian-like systems, you can install them using:
 .. code-block:: shell
 
     sudo apt-get update
-    # Install dependencies
+    # Install system dependencies
     sudo apt-get install curl python3-pip python3-venv git unzip libldap2-dev libsasl2-dev gettext-base zlib1g-dev libffi-dev libssl-dev
+
     # Funkwhale dependencies
     sudo apt install build-essential ffmpeg libjpeg-dev libmagic-dev libpq-dev postgresql-client python3-dev make
 
@@ -30,13 +46,19 @@ On Arch Linux and its derivatives:
 
 .. code-block:: shell
 
-    # Install dependencies
+    # Install system dependencies
     sudo pacman -S curl python-pip python-virtualenv git unzip
+
     # Funkwhale dependencies
     sudo pacman -S curl file ffmpeg libjpeg-turbo libpqxx python libldap libsasl
 
-Layout
+External Authentication (LDAP)
--------
+------------------------------
+
+LDAP support requires some additional dependencies to enable. On the OS level both ``libldap2-dev`` and ``libsasl2-dev`` are required, and the Python modules ``python-ldap`` and ``python-django-auth-ldap`` must be installed. These dependencies are all included in the ``requirements.*`` files so deploying with those will install these dependencies by default. However, they are not required unless LDAP support is explicitly enabled. See :doc:`../admin/ldap` for more details.
+
+Installation structure
+----------------------
 
 All Funkwhale-related files will be located under ``/srv/funkwhale`` apart
 from database files and a few configuration files. We will also have a
@@ -77,24 +99,23 @@ Create the aforementioned directories:
 
 The ``virtualenv`` directory is a bit special and will be created separately.
 
-Download latest Funkwhale release
+Download the latest Funkwhale release
-----------------------------------
+-------------------------------------
 
 Funkwhale is splitted in two components:
 
-1. The API, which will handle music storage and user accounts
+1. The API, which will handle music storage and user accounts;
-2. The frontend, that will simply connect to the API to interact with its data
+2. The frontend, that will simply connect to the API to interact with its data.
 
 Those components are packaged in subsequent releases, such as 0.1, 0.2, etc.
 You can browse the :doc:`changelog </changelog>` for a list of available releases
 and pick the one you want to install, usually the latest one should be okay.
 
-In this guide, we'll assume you want to install the latest version of Funkwhale,
+In this guide, we will assume you want to install the latest version of Funkwhale, which is |version|:
-which is |version|:
 
-First, we'll download the latest api release.
+First, we will download the latest api release:
 
-.. parsed-literal::
+.. code-block:: shell
 
     curl -L -o "api-|version|.zip" "https://dev.funkwhale.audio/funkwhale/funkwhale/-/jobs/artifacts/|version|/download?job=build_api"
     unzip "api-|version|.zip" -d extracted
@@ -102,9 +123,9 @@ First, we'll download the latest api release.
     rm -rf extracted
 
 
-Then we'll download the frontend files:
+Then we will download the frontend files:
 
-.. parsed-literal::
+.. code-block:: shell
 
     curl -L -o "front-|version|.zip" "https://dev.funkwhale.audio/funkwhale/funkwhale/-/jobs/artifacts/|version|/download?job=build_front"
     unzip "front-|version|.zip" -d extracted
@@ -114,7 +135,7 @@ Then we'll download the frontend files:
 .. note::
 
     You can also choose to get the code directly from the git repo. In this
-    case, run
+    case, run::
 
         cd /srv
 
@@ -124,7 +145,7 @@ Then we'll download the frontend files:
 
     The above clone command uses the master branch instead of the default develop branch, as master is stable and more suited for production setups.
 
-    You'll also need to re-create the folders we make earlier:
+    You'll also need to re-create the folders we make earlier::
 
         mkdir -p config data/static data/media data/music front
 
@@ -158,9 +179,7 @@ This will result in a ``virtualenv`` directory being created in
 ``/srv/funkwhale/virtualenv``.
 
 In the rest of this guide, we'll need to activate this environment to ensure
-dependencies are installed within it, and not directly on your host system.
+dependencies are installed within it, and not directly on your host system. This is done with the following command:
-
-This is done with the following command:
 
 .. code-block:: shell
 
@@ -195,7 +214,7 @@ Download the sample environment file:
 
 .. note::
 
-    if you used git to get the latest version of the code earlier, you can instead do
+    if you used git to get the latest version of the code earlier, you can instead do::
 
         cp /srv/funkwhale/deploy/env.prod.sample /srv/funkwhale/config/.env
 
@@ -215,13 +234,68 @@ configuration options are mentioned at the top of the file.
 Paste the secret key you generated earlier at the entry
 ``DJANGO_SECRET_KEY`` and populate the ``DATABASE_URL``
 and ``CACHE_URL`` values based on how you configured
-your PostgreSQL and Redis servers in
+your PostgreSQL and Redis servers in.
-:doc:`external dependencies <./external_dependencies>`.
 
 Database setup
----------------
+--------------
 
-You should now be able to import the initial database structure:
+Funkwhale requires a PostgreSQL database to work properly. Please refer
+to the `PostgreSQL documentation <https://www.postgresql.org/download/>`_
+for installation instructions specific to your os.
+
+On Debian-like systems, you would install the database server like this:
+
+.. code-block:: shell
+
+    sudo apt-get install postgresql postgresql-contrib
+
+On Arch Linux and its derivatives:
+
+.. code-block:: shell
+
+    sudo pacman -S postgresql
+
+On Arch Linux, you'll also need to initialize the database. See `the Arch Linux wiki <https://wiki.archlinux.org/index.php/Postgresql#Initial_configuration>`_.
+
+The remaining steps are heavily inspired from `this Digital Ocean guide <https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04>`_.
+
+Open a database shell:
+
+.. code-block:: shell
+
+    sudo -u postgres psql
+
+Create the project database and user:
+
+.. code-block:: shell
+
+    CREATE DATABASE funkwhale WITH ENCODING 'utf8';
+    CREATE USER funkwhale;
+    GRANT ALL PRIVILEGES ON DATABASE funkwhale TO funkwhale;
+
+.. warning::
+
+    It's important that you use utf-8 encoding for your database,
+    otherwise you'll end up with errors and crashes later on when dealing
+    with music metadata that contains non-ascii chars.
+
+Assuming you already have :ref:`created your funkwhale user <create-funkwhale-user>`,
+you should now be able to open a postgresql shell:
+
+.. code-block:: shell
+
+    sudo -u funkwhale -H psql
+
+Unless you give a superuser access to the database user, you should also
+enable some extensions on your database server, as those are required
+for Funkwhale to work properly:
+
+.. code-block:: shell
+
+    sudo -u postgres psql funkwhale -c 'CREATE EXTENSION "unaccent";'
+    sudo -u postgres psql funkwhale -c 'CREATE EXTENSION "citext";'
+
+Now that the database has been created, import the initial database structure using the virtualenv created before:
 
 .. code-block:: shell
 
@@ -246,7 +320,7 @@ This will create the required tables and rows.
 Create an admin account
 -----------------------
 
-You can then create your first user account:
+Using the virtualenv created before, create your first user account:
 
 .. code-block:: shell
 
@@ -274,9 +348,59 @@ in your ``.env`` file.
 Systemd unit file
 ------------------
 
-See :doc:`./systemd`.
+.. note::
+
+    All the command lines below should be executed as root.
+
+Systemd offers a convenient way to manage your Funkwhale instance if you're
+not using docker. We'll see how to setup systemd to properly start a Funkwhale instance.
+
+First, download the sample unitfiles:
+
+.. parsed-literal::
+
+    sudo curl -L -o "/etc/systemd/system/funkwhale.target" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale.target"
+    sudo curl -L -o "/etc/systemd/system/funkwhale-server.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-server.service"
+    sudo curl -L -o "/etc/systemd/system/funkwhale-worker.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-worker.service"
+    sudo curl -L -o "/etc/systemd/system/funkwhale-beat.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-beat.service"
+
+This will download three unitfiles:
+
+- ``funkwhale-server.service`` to launch the Funkwhale web server;
+- ``funkwhale-worker.service`` to launch the Funkwhale task worker;
+- ``funkwhale-beat.service`` to launch the Funkwhale task beat (this is for recurring tasks);
+- ``funkwhale.target`` to easily stop and start all of the services at once.
+
+You can of course review and edit them to suit your deployment scenario
+if needed, but the defaults should be fine.
+
+Once the files are downloaded, reload systemd:
+
+.. code-block:: shell
+
+    sudo systemctl daemon-reload
+
+And start the services:
+
+.. code-block:: shell
+
+    sudo systemctl start funkwhale.target
+
+To ensure all Funkwhale processes are started automatically after a reboot, run:
+
+.. code-block:: shell
+    
+    sudo systemctl enable funkwhale-server
+    sudo systemctl enable funkwhale-worker
+    sudo systemctl enable funkwhale-beat
+
+You can check the statuses of all processes like this:
+
+.. code-block:: shell
+
+    sudo systemctl status funkwhale-\*
 
 Reverse proxy setup
 --------------------
 
-See :ref:`reverse-proxy <reverse-proxy-setup>`.
+See :ref:`Reverse proxy <reverse-proxy-setup>`.

docs/installation/external_dependencies.rst

@@ -1,99 +0,0 @@
-External dependencies
-=====================
-
-
-.. note::
-
-    Those dependencies are handled automatically if you are
-    :doc:`deploying using docker <./docker>`
-
-Database setup (PostgreSQL)
----------------------------
-
-Funkwhale requires a PostgreSQL database to work properly. Please refer
-to the `PostgreSQL documentation <https://www.postgresql.org/download/>`_
-for installation instructions specific to your os.
-
-On Debian-like systems, you would install the database server like this:
-
-.. code-block:: shell
-
-    sudo apt-get install postgresql postgresql-contrib
-
-On Arch Linux and its derivatives:
-
-.. code-block:: shell
-
-    sudo pacman -S postgresql
-
-On Arch, you'll also need to initialize the database. See `the Arch Linux wiki <https://wiki.archlinux.org/index.php/Postgresql#Initial_configuration>`_.
-
-The remaining steps are heavily inspired from `this Digital Ocean guide <https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04>`_.
-
-Open a database shell:
-
-.. code-block:: shell
-
-    sudo -u postgres psql
-
-Create the project database and user:
-
-.. code-block:: shell
-
-    CREATE DATABASE "funkwhale"
-      WITH ENCODING 'utf8';
-    CREATE USER funkwhale;
-    GRANT ALL PRIVILEGES ON DATABASE funkwhale TO funkwhale;
-
-.. warning::
-
-    It's important that you use utf-8 encoding for your database,
-    otherwise you'll end up with errors and crashes later on when dealing
-    with music metadata that contains non-ascii chars.
-
-
-Assuming you already have :ref:`created your funkwhale user <create-funkwhale-user>`,
-you should now be able to open a postgresql shell:
-
-.. code-block:: shell
-
-    sudo -u funkwhale -H psql
-
-Unless you give a superuser access to the database user, you should also
-enable some extensions on your database server, as those are required
-for Funkwhale to work properly:
-
-.. code-block:: shell
-
-    sudo -u postgres psql funkwhale -c 'CREATE EXTENSION "unaccent";'
-    sudo -u postgres psql funkwhale -c 'CREATE EXTENSION "citext";'
-
-
-Cache setup (Redis)
--------------------
-
-Funkwhale also requires a cache server:
-
-- To make the whole system faster, by caching network requests or database
-  queries
-- To handle asynchronous tasks such as music import
-
-On Debian-like distributions, a redis package is available, and you can
-install it:
-
-.. code-block:: shell
-
-    sudo apt-get install redis-server
-
-On Arch Linux and its derivatives:
-
-.. code-block:: shell
-
-    sudo pacman -S redis
-
-This should be enough to have your redis server set up.
-
-External Authentication (LDAP)
-------------------------------
-
-LDAP support requires some additional dependencies to enable. On the OS level both ``libldap2-dev`` and ``libsasl2-dev`` are required, and the Python modules ``python-ldap`` and ``python-django-auth-ldap`` must be installed. These dependencies are all included in the ``requirements.*`` files so deploying with those will install these dependencies by default. However, they are not required unless LDAP support is explicitly enabled. See :doc:`./ldap` for more details.

docs/installation/index.rst

@@ -1,5 +1,5 @@
 Installation
-=============
+============
 
 Requirements
 ------------
@@ -11,12 +11,83 @@ Regardless of your chosen installation method, the following requirements must b
 
 .. note::
 
-    Because of the federated nature of Funkwhale, **we strongly recommend you not to change the Funkwhale domain after initial deployment**, as it is likely to break
+    Because of the federated nature of Funkwhale, **it is strongly recommended not to change the Funkwhale domain after initial deployment**, as it is likely to break
     your installation.
 
+Project architecture
+--------------------
+
+The project relies on the following components and services to work:
+
+- A web application server (Python/Django/Gunicorn)
+- A PostgreSQL database to store application data
+- A redis server to store cache and tasks data
+- A celery worker to run asynchronous tasks (such as music import)
+- A celery scheduler to run recurrent tasks
+- A `ntp-synced clock <https://wiki.debian.org/NTP>`_ to ensure federation is working seamlessly
+
+.. note::
+
+    The synced clock is needed for federation purpose, to assess
+    the validity of incoming requests.
+
+Hardware requirements
+---------------------
+
+Funkwhale is not especially CPU hungry. On a dockerized instance with 2 CPUs
+and a few active users, the memory footprint is around 500Mb::
+
+   CONTAINER                   MEM USAGE
+   funkwhale_api_1             202  MiB
+   funkwhale_celerybeat_1      96   MiB
+   funkwhale_celeryworker_1    168  MiB
+   funkwhale_postgres_1        22   MiB
+   funkwhale_redis_1           1    MiB
+
+Some users have reported running Funkwhale on Raspberry Pis with a memory
+consumption of less than 350 MiB.
+
+Thus, Funkwhale should run fine on commodity hardware, small hosting boxes and
+Raspberry Pi. We lack real-world examples of such deployments, so don't hesitate
+do give us your feedback (either positive or negative).
+
+Check out :doc:`../admin/optimization` for advice on how to tune your instance on small
+configurations.
+
+Software requirements
+---------------------
+
+Software requirements will vary depending of your installation method. For
+Docker-based installations, the only requirement will be an Nginx reverse-proxy
+that will expose your instance to the outside world.
+
+If you plan to install your Funkwhale instance without Docker, most of the
+dependencies should be available in your distribution's repositories.
+
+.. note::
+
+   Funkwhale works only with Python >= 3.5, as we need support for async/await.
+   Older versions of Python are not supported.
+
 Available installation methods
 -------------------------------
 
+Funkwhale can be installed using one of the following method:
+  
+- Quick install, the most straight forward way to get Funkwhale;
+- Mono-container Docker installation;
+- Multi-container Docker installation;
+- Manual Debian and Arch Linux installation;
+- `Ansible role <https://dev.funkwhale.audio/funkwhale/ansible/>`_.
+
+Further, Funkwhale packages are available for the following platforms:
+
+- `YunoHost 3 <https://yunohost.org/>`_: https://github.com/YunoHost-Apps/funkwhale_ynh (kindly maintained by `@Jibec <https://github.com/Jibec>`_)
+- ArchLinux (as an AUR package): if you'd rather use a package, check out this alternative installation method on ArchLinux: https://wiki.archlinux.org/index.php/Funkwhale (package and wiki kindly maintained by getzee)
+- `NixOS <https://github.com/mmai/funkwhale-nixos>`_ (kindly maintained by @mmai)
+- `Helm chart <https://gitlab.com/ananace/charts/>`_ to install Funkwhale on Kubernetes (kindly maintained by `@ananace <https://gitlab.com/ananace>`_)
+- `HomelabOS <https://homelabos.com/docs/software/funkwhale/>`_
+
 Quick install
 ^^^^^^^^^^^^^
 
@@ -40,85 +111,13 @@ Additional info:
 Alternative installation methods
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-We also offer Docker images, an installation guide for Debian 9 and Arch Linux, and `an
-Ansible role <https://dev.funkwhale.audio/funkwhale/ansible/>`_.
-
 .. toctree::
    :maxdepth: 1
 
-   external_dependencies
    debian
    docker
-   systemd
    non_amd64_architectures
 
-Third-party packages
-^^^^^^^^^^^^^^^^^^^^
-
-Funkwhale packages are available for the following platforms:
-
-- `YunoHost 3 <https://yunohost.org/>`_: https://github.com/YunoHost-Apps/funkwhale_ynh (kindly maintained by `@Jibec <https://github.com/Jibec>`_)
-- ArchLinux (as an AUR package): if you'd rather use a package, check out this alternative installation method on ArchLinux: https://wiki.archlinux.org/index.php/Funkwhale (package and wiki kindly maintained by getzee)
-- `NixOS <https://github.com/mmai/funkwhale-nixos>`_ (kindly maintained by @mmai)
-- `Helm chart <https://gitlab.com/ananace/charts/>`_ to install Funkwhale on Kubernetes (kindly maintained by `@ananace <https://gitlab.com/ananace>`_)
-- `HomelabOS <https://homelabos.com/docs/software/funkwhale/>`_
-
-Project architecture
---------------------
-
-The project relies on the following components and services to work:
-
-- A web application server (Python/Django/Gunicorn)
-- A PostgreSQL database to store application data
-- A redis server to store cache and tasks data
-- A celery worker to run asynchronous tasks (such as music import)
-- A celery scheduler to run recurrent tasks
-- A `ntp-synced clock <https://wiki.debian.org/NTP>`_ to ensure federation is working seamlessly
-
-.. note::
-
-    The synced clock is needed for federation purpose, to assess
-    the validity of incoming requests.
-
-
-Hardware requirements
----------------------
-
-Funkwhale is not especially CPU hungry. On a dockerized instance with 2 CPUs
-and a few active users, the memory footprint is around ~500Mb::
-
-   CONTAINER                   MEM USAGE
-   funkwhale_api_1             202.1 MiB
-   funkwhale_celerybeat_1      96.52 MiB
-   funkwhale_celeryworker_1    168.7 MiB
-   funkwhale_postgres_1        22.73 MiB
-   funkwhale_redis_1           1.496 MiB
-
-Some users have reported running Funkwhale on Raspberry Pis with a memory
-consumption of less than 350MiB.
-
-Thus, Funkwhale should run fine on commodity hardware, small hosting boxes and
-Raspberry Pi. We lack real-world examples of such deployments, so don't hesitate
-do give us your feedback (either positive or negative).
-
-Check out :doc:`optimization` for advice on how to tune your instance on small
-configurations.
-
-Software requirements
----------------------
-
-Software requirements will vary depending of your installation method. For
-Docker-based installations, the only requirement will be an Nginx reverse-proxy
-that will expose your instance to the outside world.
-
-If you plan to install your Funkwhale instance without Docker, most of the
-dependencies should be available in your distribution's repositories.
-
-.. note::
-
-   Funkwhale works only with Python >= 3.5, as we need support for async/await.
-   Older versions of Python are not supported.
-
 Running Funkwhale on the develop branch
 ---------------------------------------
 
@@ -144,18 +143,20 @@ since you may have to apply manual actions for your instance to continue to work
 
 .. _frontend-setup:
 
-Frontend setup
+Serving only the frontend
----------------
+-------------------------
 
 .. note::
 
     You do not need to do this if you are deploying using Docker, as frontend files
     are already included in the docker image.
 
+    You also do not need to do this if you are deploying manually on Debian or Arch, as this is covered by the corresponding documentation already.
+
 
 Files for the web frontend are purely static and can simply be downloaded, unzipped and served from any webserver:
 
-.. parsed-literal::
+.. code-block:: shell
 
     cd /srv/funkwhale
     curl -L -o front.zip "https://dev.funkwhale.audio/funkwhale/funkwhale/builds/artifacts/|version|/download?job=build_front"
@@ -163,24 +164,32 @@ Files for the web frontend are purely static and can simply be downloaded, unzip
 
 .. _reverse-proxy-setup:
 
-Reverse proxy
+Reverse proxy configuration
---------------
+---------------------------
 
 In order to make Funkwhale accessible from outside your server and to play nicely with other applications on your machine, you should configure a reverse proxy.
 
+We offer sample configurations for Nginx, Apache2 and Caddy.
+
+.. note::
+
+    You can freely adapt the proposed configuration to your own needs, as we cannot
+    cover every use case with a single template, especially when it's related
+    to SSL configuration.
+
 Nginx
 ^^^^^
 
 Ensure you have a recent version of nginx on your server. On Debian-like system, you would have to run the following:
 
-.. code-block:: bash
+.. code-block:: shell
 
     sudo apt-get update
     sudo apt-get install nginx
 
 On Arch Linux and its derivatives:
 
-.. code-block:: bash
+.. code-block:: shell
 
     sudo pacman -S nginx
 
@@ -188,11 +197,15 @@ To avoid configuration errors at this level, we will generate an nginx configura
 using your .env file. This will ensure your reverse-proxy configuration always
 match the application configuration and make upgrade/maintenance easier.
 
+.. note::
+    The following commands need to be run as superuser.
+
 On docker deployments, run the following commands:
 
-.. parsed-literal::
+.. code-block:: shell
 
     export FUNKWHALE_VERSION="|version|"
+
     # download the needed files
     curl -L -o /etc/nginx/funkwhale_proxy.conf "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale_proxy.conf"
     curl -L -o /etc/nginx/sites-available/funkwhale.template "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/docker.proxy.template"
@@ -209,7 +222,6 @@ On docker deployments, run the following commands:
 
 On non-docker deployments, run the following commands:
 
-
 .. parsed-literal::
 
     export FUNKWHALE_VERSION="|version|"
@@ -235,12 +247,6 @@ On non-docker deployments, run the following commands:
 
         grep '${' /etc/nginx/sites-available/funkwhale.conf
 
-.. note::
-
-    You can freely adapt the resulting file to your own needs, as we cannot
-    cover every use case with a single template, especially when it's related
-    to SSL configuration.
-
 Finally, enable the resulting configuration:
 
 .. code-block:: bash
@@ -249,38 +255,30 @@ Finally, enable the resulting configuration:
 
 .. warning::
 
-    If you plan to use to in-place import, ensure the alias value
+    If you plan to use in-place import, ensure the alias value
     in the ``_protected/music`` location matches your MUSIC_DIRECTORY_SERVE_PATH
     env var.
 
-HTTPS Configuration
+Finally, check that the configuration is valid with ``nginx -t`` then reload your nginx server with ``sudo systemctl reload nginx``.
-:::::::::::::::::::
 
-At this point you will need a SSL certificate to enable HTTPS on your server.
+.. note::
-The default nginx configuration assumes you have those available at ``/etc/letsencrypt/live/${FUNKWHALE_HOSTNAME}/``, which
+    Music (and other static) files are never served by the app itself, but by the reverse
-is the path used by `certbot <https://certbot.eff.org/docs/>`_ when generating certificates with Let's Encrypt.
+    proxy. This is needed because a webserver is way more efficient at serving
+    files than a Python process.
 
-In you already have a certificate you'd like to use, simply update the nginx configuration
+    However, we do want to ensure users have the right to access music files, and
-and replace ``ssl_certificate`` and ``ssl_certificate_key`` values with the proper paths.
+    it can't be done at the proxy's level. To tackle this issue, `we use
+    nginx's internal directive <http://nginx.org/en/docs/http/ngx_http_core_module.html#internal>`_.
 
-If you don't have one, comment or remove the lines starting with ``ssl_certificate`` and ``ssl_certificate_key``. You can then proceed to generate
+    When the API receives a request on its music serving endpoint, it will check
-a certificate, as shown below:
+    that the user making the request can access the file. Then, it will return an empty
-
+    response with a ``X-Accel-Redirect`` header. This header will contain the path
-.. code-block:: shell
+    to the file to serve to the user, and will be picked by nginx, but never sent
-
+    back to the client.
-    # install certbot with nginx support
-    sudo apt install python-certbot-nginx
-    # generate the certificate
-    # (accept the terms of service if prompted)
-    sudo certbot --nginx -d yourfunkwhale.domain
-
-This should create a valid certificate and edit the nginx configuration to use the new certificate.
-
-Reloading
-:::::::::
-
-Check the configuration is valid with ``nginx -t`` then reload your nginx server with ``sudo systemctl reload nginx``.
 
+    Using this technique, we can ensure music files are covered by the authentication
+    and permission policy of your instance, while remaining as performant
+    as possible.
 
 Apache2
 ^^^^^^^
@@ -312,11 +310,6 @@ You can tweak the configuration file according to your setup, especially the
 TLS configuration. Otherwise, defaults should work if you followed the
 installation guide.
 
-.. note::
-
-    To obtain a certificate to enable HTTPS on your server, please refer to the note in
-    the nginx chapter above.
-
 Check the configuration is valid with ``apache2ctl configtest``, and once you're
 done, load the new configuration with ``service apache2 restart``.
 
@@ -342,24 +335,36 @@ Caddy v1::
         }
     }
 
+HTTPS configuration
+^^^^^^^^^^^^^^^^^^^
 
-About internal locations
+After configuring the reverse proxy, you need a SSL certificate to enable HTTPS on your server.
-^^^^^^^^^^^^^^^^^^^^^^^^
 
-Music (and other static) files are never served by the app itself, but by the reverse
+The default reverse proxy configuration assumes you have those available at ``/etc/letsencrypt/live/${FUNKWHALE_HOSTNAME}/``, which
-proxy. This is needed because a webserver is way more efficient at serving
+is the path used by `certbot <https://certbot.eff.org/docs/>`_ when generating certificates with Let's Encrypt.
-files than a Python process.
 
-However, we do want to ensure users have the right to access music files, and
+If you already have a certificate you would like to use, simply update the reverse proxy configuration
-it can't be done at the proxy's level. To tackle this issue, `we use
+and replace the following values with the proper paths:
-nginx's internal directive <http://nginx.org/en/docs/http/ngx_http_core_module.html#internal>`_.
+- For nginx: ``ssl_certificate`` and ``ssl_certificate_key``;
+- For Apache2: ``SSLCertificateFile`` and ``SSLCertificateKeyFile``.
+
+If you don't have one, comment or remove the lines starting with ``ssl_certificate`` and ``ssl_certificate_key`` for nginx, and ``SSLCertificateFile`` and ``SSLCertificateKeyFile`` for Apache2. You can then proceed to generate
+a certificate, as shown below. These instructions are provided by `certbot <https://certbot.eff.org/instructions>`:
+
+.. code-block:: shell
+
+    # install certbot
+    sudo snap install core; sudo snap refresh core
+    sudo snap install --classic certbot
+    sudo ln -s /snap/bin/certbot /usr/bin/certbot
+
+    # if you are using nginx: generate the certificate
+    sudo certbot --nginx -d yourfunkwhale.domain
+
+    # if you are using Apache2: generate the certificate
+    sudo certbot --apache -d yourfunkwhale.domain
+
+This creates a valid certificate and edit the nginx or Apache2 configuration to use the new certificate. The certificate will be automatically renewed when they expire.
+    
 
-When the API receives a request on its music serving endpoint, it will check
-that the user making the request can access the file. Then, it will return an empty
-response with a ``X-Accel-Redirect`` header. This header will contain the path
-to the file to serve to the user, and will be picked by nginx, but never sent
-back to the client.
 
-Using this technique, we can ensure music files are covered by the authentication
-and permission policy of your instance, while remaining as performant
-as possible.

docs/installation/non_amd64_architectures.rst

@@ -42,7 +42,7 @@ guide.
 
 
 Building the mono-container Docker image (funkwhale/all-in-one)
---------------------------------------------------------------
+---------------------------------------------------------------
 
 This image is intended to be used in conjunction with our :ref:`Mono-container installation guide <docker-mono-container>`.
 guide.

docs/installation/systemd.rst

@@ -1,56 +0,0 @@
-Systemd configuration
-----------------------
-
-.. note::
-
-    All the command lines below should be executed as root.
-
-Systemd offers a convenient way to manage your Funkwhale instance if you're
-not using docker.
-
-We'll see how to setup systemd to properly start a Funkwhale instance.
-
-First, download the sample unitfiles:
-
-.. parsed-literal::
-
-    curl -L -o "/etc/systemd/system/funkwhale.target" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale.target"
-    curl -L -o "/etc/systemd/system/funkwhale-server.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-server.service"
-    curl -L -o "/etc/systemd/system/funkwhale-worker.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-worker.service"
-    curl -L -o "/etc/systemd/system/funkwhale-beat.service" "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-beat.service"
-
-This will download three unitfiles:
-
-- ``funkwhale-server.service`` to launch the Funkwhale web server
-- ``funkwhale-worker.service`` to launch the Funkwhale task worker
-- ``funkwhale-beat.service`` to launch the Funkwhale task beat (this is for recurring tasks)
-- ``funkwhale.target`` to easily stop and start all of the services at once
-
-You can of course review and edit them to suit your deployment scenario
-if needed, but the defaults should be fine.
-
-Once the files are downloaded, reload systemd:
-
-.. code-block:: shell
-
-    systemctl daemon-reload
-
-And start the services:
-
-.. code-block:: shell
-
-    systemctl start funkwhale.target
-
-To ensure all Funkwhale processes are started automatically after a reboot, run:
-
-.. code-block:: shell
-    
-    systemctl enable funkwhale-server
-    systemctl enable funkwhale-worker
-    systemctl enable funkwhale-beat
-
-You can check the statuses of all processes like this:
-
-.. code-block:: shell
-
-    systemctl status funkwhale-\*

docs/users/cli.rst

@@ -16,8 +16,6 @@ Here is a (non-exhaustive) list of operations you can perform via the CLI:
 
 .. contents:: Table of Contents
 
-
-
 Installation
 ------------
 
@@ -54,13 +52,6 @@ Here are a couple of commands you can try to get started:
     # Search artists matching "zebra" on open.audio
     funkwhale -H https://open.audio artists ls "zebra"
 
-More examples
--------------
-
-You should find enough in this reference document to start using the CLI on your own.
-
-However, we've compiled :doc:`a list of example uses of the CLI <examples>` with advice and explanations, if you want to check it out ;)
-
 Getting help
 ------------
 
@@ -126,8 +117,8 @@ List of configuration options
 | ``-q``, ``--quiet``                  | ``FUNKWHALE_QUIET``                            | ``true``                                   | Completely disable logging                                    |
 +--------------------------------------+------------------------------------------------+--------------------------------------------+---------------------------------------------------------------+
 
-Listing results
+Read commands
----------------
+-------------
 
 All commands that list results - such as ``funkwhale albums ls`` or ``funkwhale tracks ls`` - share similar behaviors and sets of arguments.
 
@@ -238,8 +229,11 @@ The ``-i`` or ``--ids`` flag displays only the IDs of results, one per line::
 This is especially useful in conjunction with other commands (like deletion commands) and piping.
 Note that this is also technically equivalent to applying the ``--no-headers``, ``--format plain`` and ``--column ID`` flags.
 
+Write commands
+--------------
+
 Deleting objects
-----------------
+^^^^^^^^^^^^^^^^
 
 Some resources support deletion, via commands such as ``funkwhale libraries rm`` or ``funkwhale playlists rm``, followed by one or more IDs::
 
@@ -250,3 +244,68 @@ By default, the ``rm`` command will ask for confirmation, but you can disable th
 
 
 .. _API Documentation: https://docs.funkwhale.audio/swagger/
+
+Examples
+--------
+
+Uploading local files
+^^^^^^^^^^^^^^^^^^^^^
+
+**Goal**: create a library and upload all MP3 files from ``~/Music`` to it
+
+**Commands**::
+
+    funkwhale libraries create --name "My awesome library" --visibility me
+    # copy the returned UUID
+    funkwhale uploads create <UUID> ~/Music/**/*.mp3
+
+
+Favorite an entire album
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+**Goal**: retrieve all the tracks from an album and add these to your favorites
+
+**Commands**::
+
+    # retrieve the album ID
+    funkwhale albums ls "The Slip"
+
+    # Copy the ID, then retrieve 100 pages of tracks from that album
+    # get only the IDs and pipe those to the favorite creation command
+    funkwhale tracks ls -f "album=<ID>" --ids --limit 100 \
+        | xargs funkwhale favorites tracks create
+
+
+Mirror an artist discography locally
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+**Goal**: Download the discography of an artist locally, in the ``~/Music`` directory, in an ``Artist/Album/Track`` folder hierarchy
+
+**Commands**::
+
+    # retrieve the artist ID
+    funkwhale artists ls "Nine Inch Nails"
+
+    # Copy the ID, then retrieve 100 pages of tracks from that artist
+    # get only the IDs and pipe those to the download command
+    funkwhale tracks ls -f "artist=<ID>" --ids --limit 100 \
+        | xargs funkwhale tracks download \
+            -f mp3 -d ~/Music -t "{artist}/{album}/{title}.{extension}"
+
+
+Open a remote album in VLC
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+**Goal**: Variation of the previous example, but instead of downloading an artist discography, we listen to an album in VLC
+
+**Commands**::
+
+    # retrieve the album ID
+    funkwhale albums ls "The Slip"
+
+    # Copy the ID, then retrieve 100 pages of tracks from that album
+    # get only the IDs, download the corresponding tracks and pipe the audio
+    # directly to VLC
+    funkwhale tracks ls -f "album=<ID>" --ids --limit 100 \
+        | xargs funkwhale tracks download \
+        | vlc -

docs/users/index.rst

@@ -33,7 +33,7 @@ Using Funkwhale
    follow
    apps
    reports
-   ../cli/index
+   cli
 
 Troubleshooting Issues
 ----------------------

docs/users/upload.rst

@@ -3,11 +3,11 @@ Uploading Content To Funkwhale
 
 To upload content to any Funkwhale instance, you need:
 
-1. :doc:`An account on that instance <create>`
+1. :doc:`An account on that instance <create>`;
-2. :ref:`Storage space <upload-storage>`
+2. :ref:`Storage space <upload-storage>`;
-3. :ref:`A library <upload-library>` or :ref:`channel <upload-channel>`
+3. :ref:`A library <upload-library>` or :ref:`channel <upload-channel>`;
-4. :ref:`Properly tagged files <upload-tagging>`
+4. :ref:`Properly tagged files <upload-tagging>`;
-5. :ref:`To upload your files <upload-upload>`
+5. To upload your files :ref:`to a library <upload_library>` or :ref:`to a channel <channel_upload>`.
 
 .. _upload-storage:
 
@@ -93,14 +93,14 @@ However, Funkwhale can understand and use additional tags to enhance user experi
 | ``Artist`` (required)            | ``Nine Inch Nails``                        | The artist name                                               |
 |                                  |                                            |                                                               |
 +----------------------------------+--------------------------------------------+---------------------------------------------------------------+
-| ``Album``                        | ``The Slip``                               | The album title. If none is provided, an [Unknown Album]     |
+| ``Album``                        | ``The Slip``                               | The album title. If none is provided, an [Unknown Album]      |
 |                                  |                                            | entry will be created                                         |
 +----------------------------------+--------------------------------------------+---------------------------------------------------------------+
 | ``Album artist``                 | ``Trent Reznor``                           | The album artist name (can be different than the track        |
 |                                  |                                            | artist)                                                       |
 |                                  |                                            |                                                               |
 +----------------------------------+--------------------------------------------+---------------------------------------------------------------+
-| ``Genre``                        | ``Industrial, Metal``                      | A comma separated list of tags to associate with the track     |
+| ``Genre``                        | ``Industrial, Metal``                      | A comma separated list of tags to associate with the track    |
 |                                  |                                            | Other supported separators: ``;`` and ``/``                   |
 +----------------------------------+--------------------------------------------+---------------------------------------------------------------+
 | ``Track number``                 | ``4``                                      | The position of the track in the album/release                |