Merge branch 'contributing-documentation' into 'develop'
Contributing documentation See merge request funkwhale/funkwhale!178
This commit is contained in:
commit
fc352b5e52
|
@ -0,0 +1,442 @@
|
||||||
|
Contibute to Funkwhale development
|
||||||
|
==================================
|
||||||
|
|
||||||
|
First of all, thank you for your interest in the project! We really
|
||||||
|
appreciate the fact that you're about to take some time to read this
|
||||||
|
and hack on the project.
|
||||||
|
|
||||||
|
This document will guide you through common operations such as:
|
||||||
|
|
||||||
|
- Setup your development environment
|
||||||
|
- Working on your first issue
|
||||||
|
- Writing unit tests to validate your work
|
||||||
|
- Submit your work
|
||||||
|
|
||||||
|
|
||||||
|
Setup your development environment
|
||||||
|
----------------------------------
|
||||||
|
|
||||||
|
If you want to fix a bug or implement a feature, you'll need
|
||||||
|
to run a local, development copy of funkwhale.
|
||||||
|
|
||||||
|
We provide a docker based development environment, which should
|
||||||
|
be both easy to setup and work similarly regardless of your
|
||||||
|
development machine setup.
|
||||||
|
|
||||||
|
Instructions for bare-metal setup will come in the future (Merge requests
|
||||||
|
are welcome).
|
||||||
|
|
||||||
|
Installing docker and docker-compose
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
This is already cover in the relevant documentations:
|
||||||
|
|
||||||
|
- https://docs.docker.com/install/
|
||||||
|
- https://docs.docker.com/compose/install/
|
||||||
|
|
||||||
|
Cloning the project
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Visit https://code.eliotberriot.com/funkwhale/funkwhale and clone the repository using SSH or HTTPS. Exemple using SSH::
|
||||||
|
|
||||||
|
git clone ssh://git@code.eliotberriot.com:2222/funkwhale/funkwhale.git
|
||||||
|
cd funkwhale
|
||||||
|
|
||||||
|
|
||||||
|
A note about branches
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Next release development occurs on the "develop" branch, and releases are made on the "master" branch. Therefor, when submitting Merge Requests, ensure you are merging on the develop branch.
|
||||||
|
|
||||||
|
|
||||||
|
Working with docker
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
In developpement, we use the docker-compose file named ``dev.yml``, and this is why all our docker-compose commands will look like this::
|
||||||
|
|
||||||
|
docker-compose -f dev.yml logs
|
||||||
|
|
||||||
|
If you do not want to add the ``-f dev.yml`` snippet everytime, you can run this command before starting your work::
|
||||||
|
|
||||||
|
export COMPOSE_FILE=dev.yml
|
||||||
|
|
||||||
|
|
||||||
|
Building the containers
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
On your initial clone, or if there have been some changes in the
|
||||||
|
app dependencies, you will have to rebuild your containers. This is done
|
||||||
|
via the following command::
|
||||||
|
|
||||||
|
docker-compose -f dev.yml build
|
||||||
|
|
||||||
|
|
||||||
|
Creating your env file
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
We provide a working .env.dev configuration file that is suitable for
|
||||||
|
development. However, to enable customization on your machine, you should
|
||||||
|
also create a .env file that will hold your personal environment
|
||||||
|
variables (those will not be commited to the project).
|
||||||
|
|
||||||
|
Create it like this::
|
||||||
|
|
||||||
|
touch .env
|
||||||
|
|
||||||
|
|
||||||
|
Database management
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
To setup funkwhale's database schema, run this::
|
||||||
|
|
||||||
|
docker-compose -f dev.yml run --rm api python manage.py migrate
|
||||||
|
|
||||||
|
This will create all the tables needed for the API to run proprely.
|
||||||
|
You will also need to run this whenever changes are made on the database
|
||||||
|
schema.
|
||||||
|
|
||||||
|
It is safe to run this command multiple times, so you can run it whenever
|
||||||
|
you fetch develop.
|
||||||
|
|
||||||
|
|
||||||
|
Development data
|
||||||
|
^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
You'll need at least an admin user and some artists/tracks/albums to work
|
||||||
|
locally.
|
||||||
|
|
||||||
|
Create an admin user with the following command::
|
||||||
|
|
||||||
|
docker-compose -f dev.yml run --rm api python manage.py createsuperuser
|
||||||
|
|
||||||
|
Injecting fake data is done by running the fllowing script::
|
||||||
|
|
||||||
|
artists=25
|
||||||
|
command="from funkwhale_api.music import fake_data; fake_data.create_data($artists)"
|
||||||
|
echo $command | docker-compose -f dev.yml run --rm api python manage.py shell -i python
|
||||||
|
|
||||||
|
The previous command will create 25 artists with random albums, tracks
|
||||||
|
and metadata.
|
||||||
|
|
||||||
|
|
||||||
|
Launch all services
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Then you can run everything with::
|
||||||
|
|
||||||
|
docker-compose -f dev.yml up
|
||||||
|
|
||||||
|
This will launch all services, and output the logs in your current terminal window.
|
||||||
|
If you prefer to launch them in the background instead, use the ``-d`` flag, and access the logs when you need it via ``docker-compose -f dev.yml logs --tail=50 --follow``.
|
||||||
|
|
||||||
|
Once everything is up, you can access the various funkwhale's components:
|
||||||
|
|
||||||
|
- The Vue webapp, on http://localhost:8080
|
||||||
|
- The API, on http://localhost:8080/api/v1/
|
||||||
|
- The django admin, on http://localhost:8080/api/admin/
|
||||||
|
|
||||||
|
Stopping everything
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Once you're down with your work, you can stop running containers, if any, with::
|
||||||
|
|
||||||
|
docker-compose -f dev.yml stop
|
||||||
|
|
||||||
|
|
||||||
|
Removing everything
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
If you want to wipe your development environment completely (e.g. if you want to start over from scratch), just run::
|
||||||
|
|
||||||
|
docker-compose -f dev.yml down -v
|
||||||
|
|
||||||
|
This will wipe your containers and data, so please be careful before running it.
|
||||||
|
|
||||||
|
You can keep your data by removing the ``-v`` flag.
|
||||||
|
|
||||||
|
|
||||||
|
Working with federation locally
|
||||||
|
-------------------------------
|
||||||
|
|
||||||
|
This is not needed unless you need to work on federation-related features.
|
||||||
|
|
||||||
|
To achieve that, you'll need:
|
||||||
|
|
||||||
|
1. to update your dns resolver to resolve all your .dev hostnames locally
|
||||||
|
2. a reverse proxy (such as traefik) to catch those .dev requests and
|
||||||
|
and with https certificate
|
||||||
|
3. two instances (or more) running locally, following the regular dev setup
|
||||||
|
|
||||||
|
Resolve .dev names locally
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
If you use dnsmasq, this is as simple as doing::
|
||||||
|
|
||||||
|
echo "address=/test/172.17.0.1" | sudo tee /etc/dnsmasq.d/test.conf
|
||||||
|
sudo systemctl restart dnsmasq
|
||||||
|
|
||||||
|
If you use NetworkManager with dnsmasq integration, use this instead::
|
||||||
|
|
||||||
|
echo "address=/test/172.17.0.1" | sudo tee /etc/NetworkManager/dnsmasq.d/test.conf
|
||||||
|
sudo systemctl restart NetworkManager
|
||||||
|
|
||||||
|
Add wildcard certificate to the trusted certificates
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Simply copy bundled certificates::
|
||||||
|
|
||||||
|
sudo cp docker/ssl/test.crt /usr/local/share/ca-certificates/
|
||||||
|
sudo update-ca-certificates
|
||||||
|
|
||||||
|
This certificate is a wildcard for ``*.funkwhale.test``
|
||||||
|
|
||||||
|
Run a reverse proxy for your instances
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
|
||||||
|
Create docker network
|
||||||
|
^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Create the federation network::
|
||||||
|
|
||||||
|
docker network create federation
|
||||||
|
|
||||||
|
Launch everything
|
||||||
|
^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Launch the traefik proxy::
|
||||||
|
|
||||||
|
docker-compose -f docker/traefik.yml up -d
|
||||||
|
|
||||||
|
Then, in separate terminals, you can setup as many different instances as you
|
||||||
|
need::
|
||||||
|
|
||||||
|
export COMPOSE_PROJECT_NAME=node2
|
||||||
|
docker-compose -f dev.yml run --rm api python manage.py migrate
|
||||||
|
docker-compose -f dev.yml run --rm api python manage.py createsuperuser
|
||||||
|
docker-compose -f dev.yml up nginx api front nginx api celeryworker
|
||||||
|
|
||||||
|
Note that by default, if you don't export the COMPOSE_PROJECT_NAME,
|
||||||
|
we will default to node1 as the name of your instance.
|
||||||
|
|
||||||
|
Assuming your project name is ``node1``, your server will be reachable
|
||||||
|
at ``https://node1.funkwhale.test/``. Not that you'll have to trust
|
||||||
|
the SSL Certificate as it's self signed.
|
||||||
|
|
||||||
|
When working on federation with traefik, ensure you have this in your ``env``::
|
||||||
|
|
||||||
|
# This will ensure we don't bind any port on the host, and thus enable
|
||||||
|
# multiple instances of funkwhale to be spawned concurrently.
|
||||||
|
WEBPACK_DEVSERVER_PORT_BINDING=
|
||||||
|
# This disable certificate verification
|
||||||
|
EXTERNAL_REQUESTS_VERIFY_SSL=false
|
||||||
|
# this ensure you don't have incorrect urls pointing to http resources
|
||||||
|
FUNKWHALE_PROTOCOL=https
|
||||||
|
|
||||||
|
|
||||||
|
Typical workflow for a contribution
|
||||||
|
-----------------------------------
|
||||||
|
|
||||||
|
0. Fork the project if you did not already or if you do not have access to the main repository
|
||||||
|
1. Checkout the development branch and pull most recent changes: ``git checkout develop && git pull``
|
||||||
|
2. If working on an issue, assign yourself to the issue. Otherwise, consider open an issue before starting to work on something, especially for new features.
|
||||||
|
3. Create a dedicated branch for your work ``42-awesome-fix``. It is good practice to prefix your branch name with the ID of the issue you are solving.
|
||||||
|
4. Work on your stuff
|
||||||
|
5. Commit small, atomic changes to make it easier to review your contribution
|
||||||
|
6. Add a changelog fragment to summarize your changes: ``echo "Implemented awesome stuff (#42)" > changes/changelog.d/42.feature"``
|
||||||
|
7. Push your branch
|
||||||
|
8. Create your merge request
|
||||||
|
9. Take a step back and enjoy, we're really grateful you did all of this and took the time to contribute!
|
||||||
|
|
||||||
|
|
||||||
|
Internationalization
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
When working on the front-end, any end-user string should be translated
|
||||||
|
using either ``<i18next path="yourstring">`` or the ``$t('yourstring')``
|
||||||
|
function.
|
||||||
|
|
||||||
|
Extraction is done by calling ``yarn run i18n-extract``, which
|
||||||
|
will pull all the strings from source files and put them in a PO file.
|
||||||
|
|
||||||
|
Contributing to the API
|
||||||
|
-----------------------
|
||||||
|
|
||||||
|
Project structure
|
||||||
|
^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
.. code-block:: shell
|
||||||
|
|
||||||
|
tree api -L 2 -d
|
||||||
|
api
|
||||||
|
├── config # configuration directory (settings, urls, wsgi server)
|
||||||
|
│ └── settings # Django settings files
|
||||||
|
├── funkwhale_api # project directory, all funkwhale logic is here
|
||||||
|
├── requirements # python requirements files
|
||||||
|
└── tests # test files, matches the structure of the funkwhale_api directory
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
Unless trivial, API contributions must include unittests to ensure
|
||||||
|
your fix or feature is working as expected and won't break in the future
|
||||||
|
|
||||||
|
Running tests
|
||||||
|
^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
To run the pytest test suite, use the following command::
|
||||||
|
|
||||||
|
docker-compose -f dev.yml run --rm api pytest
|
||||||
|
|
||||||
|
This is regular pytest, so you can use any arguments/options that pytest usually accept::
|
||||||
|
|
||||||
|
# get some help
|
||||||
|
docker-compose -f dev.yml run --rm api pytest -h
|
||||||
|
# Stop on first failure
|
||||||
|
docker-compose -f dev.yml run --rm api pytest -x
|
||||||
|
# Run a specific test file
|
||||||
|
docker-compose -f dev.yml run --rm api pytest tests/test_acoustid.py
|
||||||
|
|
||||||
|
Writing tests
|
||||||
|
^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Although teaching you how to write unit tests is outside of the scope of this
|
||||||
|
document, you'll find below a collection of tips, snippets and resources
|
||||||
|
you can use if you want to learn on that subject.
|
||||||
|
|
||||||
|
Useful links:
|
||||||
|
|
||||||
|
- `A quick introduction to unit test writing with pytest <https://semaphoreci.com/community/tutorials/testing-python-applications-with-pytest>`_
|
||||||
|
- `A complete guide to Test-Driven Development (although not using Pytest) <https://www.obeythetestinggoat.com/>`_
|
||||||
|
- `pytest <https://docs.pytest.org/en/latest/>`_: documentation of our testing engine and runner
|
||||||
|
- `pytest-mock <https://pypi.org/project/pytest-mock/>`_: project page of our mocking engine
|
||||||
|
- `factory-boy <http://factoryboy.readthedocs.io/>`_: documentation of factory-boy, which we use to easily generate fake objects and data
|
||||||
|
|
||||||
|
Recommendations:
|
||||||
|
|
||||||
|
- Test files must target a module and mimic ``funkwhale_api`` directory structure: if you're writing tests for ``funkwhale_api/myapp/views.py``, you should put thoses tests in ``tests/myapp/test_views.py``
|
||||||
|
- Tests should be small and test one thing. If you need to test multiple things, write multiple tests.
|
||||||
|
|
||||||
|
We provide a lot of utils and fixtures to make the process of writing tests as
|
||||||
|
painless as possible. You'll find some usage examples below.
|
||||||
|
|
||||||
|
Use factories to create arbitrary objects:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
# funkwhale_api/myapp/users.py
|
||||||
|
|
||||||
|
def downgrade_user(user):
|
||||||
|
"""
|
||||||
|
A simple function that remove superuser status from users
|
||||||
|
and return True if user was actually downgraded
|
||||||
|
"""
|
||||||
|
downgraded = user.is_superuser
|
||||||
|
user.is_superuser = False
|
||||||
|
user.save()
|
||||||
|
return downgraded
|
||||||
|
|
||||||
|
# tests/myapp/test_users.py
|
||||||
|
from funkwhale_api.myapp import users
|
||||||
|
|
||||||
|
def test_downgrade_superuser(factories):
|
||||||
|
user = factories['users.User'](is_superuser=True)
|
||||||
|
downgraded = users.downgrade_user(user)
|
||||||
|
|
||||||
|
assert downgraded is True
|
||||||
|
assert user.is_superuser is False
|
||||||
|
|
||||||
|
def test_downgrade_normal_user_does_nothing(factories):
|
||||||
|
user = factories['users.User'](is_superuser=False)
|
||||||
|
downgraded = something.downgrade_user(user)
|
||||||
|
|
||||||
|
assert downgraded is False
|
||||||
|
assert user.is_superuser is False
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
We offer factories for almost if not all models. Factories are located
|
||||||
|
in a ``factories.py`` file inside each app.
|
||||||
|
|
||||||
|
Mocking: mocking is the process of faking some logic in our code. This is
|
||||||
|
useful when testing components that depend on each other:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
# funkwhale_api/myapp/notifications.py
|
||||||
|
|
||||||
|
def notify(email, message):
|
||||||
|
"""
|
||||||
|
A function that sends an email to the given recipient
|
||||||
|
with the given message
|
||||||
|
"""
|
||||||
|
|
||||||
|
# our email sending logic here
|
||||||
|
# ...
|
||||||
|
|
||||||
|
# funkwhale_api/myapp/users.py
|
||||||
|
from . import notifications
|
||||||
|
|
||||||
|
def downgrade_user(user):
|
||||||
|
"""
|
||||||
|
A simple function that remove superuser status from users
|
||||||
|
and return True if user was actually downgraded
|
||||||
|
"""
|
||||||
|
downgraded = user.is_superuser
|
||||||
|
user.is_superuser = False
|
||||||
|
user.save()
|
||||||
|
if downgraded:
|
||||||
|
notifications.notify(user.email, 'You have been downgraded!')
|
||||||
|
return downgraded
|
||||||
|
|
||||||
|
# tests/myapp/test_users.py
|
||||||
|
def test_downgrade_superuser_sends_email(factories, mocker):
|
||||||
|
"""
|
||||||
|
Your downgrade logic is already tested, however, we want to ensure
|
||||||
|
an email is sent when user is downgraded, but we don't have any email
|
||||||
|
server available in our testing environment. Thus, we need to mock
|
||||||
|
the email sending process.
|
||||||
|
"""
|
||||||
|
mocked_notify = mocker.patch('funkwhale_api.myapp.notifications.notify')
|
||||||
|
user = factories['users.User'](is_superuser=True)
|
||||||
|
users.downgrade_user(user)
|
||||||
|
|
||||||
|
# here, we ensure our notify function was called with proper arguments
|
||||||
|
mocked_notify.assert_called_once_with(user.email, 'You have been downgraded')
|
||||||
|
|
||||||
|
|
||||||
|
def test_downgrade_not_superuser_skips_email(factories, mocker):
|
||||||
|
mocked_notify = mocker.patch('funkwhale_api.myapp.notifications.notify')
|
||||||
|
user = factories['users.User'](is_superuser=True)
|
||||||
|
users.downgrade_user(user)
|
||||||
|
|
||||||
|
# here, we ensure no email was sent
|
||||||
|
mocked_notify.assert_not_called()
|
||||||
|
|
||||||
|
Views: you can find some readable views tests in :file:`tests/users/test_views.py`
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
A complete list of available-fixtures is available by running
|
||||||
|
``docker-compose -f dev.yml run --rm api pytest --fixtures``
|
||||||
|
|
||||||
|
|
||||||
|
Contributing to the front-end
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
Running tests
|
||||||
|
^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
To run the front-end test suite, use the following command::
|
||||||
|
|
||||||
|
docker-compose -f dev.yml run --rm front yarn run unit
|
||||||
|
|
||||||
|
We also support a "watch and test" mode were we continually relaunch
|
||||||
|
tests when changes are recorded on the file system::
|
||||||
|
|
||||||
|
docker-compose -f dev.yml run --rm front yarn run unit-watch
|
||||||
|
|
||||||
|
The latter is especially useful when you are debugging failing tests.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
The front-end test suite coverage is still pretty low
|
282
README.rst
282
README.rst
|
@ -15,282 +15,8 @@ We offer various Matrix.org rooms to discuss about funkwhale:
|
||||||
|
|
||||||
Please join those rooms if you have any questions!
|
Please join those rooms if you have any questions!
|
||||||
|
|
||||||
Running the development version
|
Contribute
|
||||||
-------------------------------
|
----------
|
||||||
|
|
||||||
If you want to fix a bug or implement a feature, you'll need
|
Contribution guidelines as well as development installation instructions
|
||||||
to run a local, development copy of funkwhale.
|
are outlined in `CONTRIBUTING <CONTRIBUTING>`_
|
||||||
|
|
||||||
We provide a docker based development environment, which should
|
|
||||||
be both easy to setup and work similarly regardless of your
|
|
||||||
development machine setup.
|
|
||||||
|
|
||||||
Instructions for bare-metal setup will come in the future (Merge requests
|
|
||||||
are welcome).
|
|
||||||
|
|
||||||
Installing docker and docker-compose
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
This is already cover in the relevant documentations:
|
|
||||||
|
|
||||||
- https://docs.docker.com/install/
|
|
||||||
- https://docs.docker.com/compose/install/
|
|
||||||
|
|
||||||
Cloning the project
|
|
||||||
^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
Visit https://code.eliotberriot.com/funkwhale/funkwhale and clone the repository using SSH or HTTPS. Exemple using SSH::
|
|
||||||
|
|
||||||
git clone ssh://git@code.eliotberriot.com:2222/funkwhale/funkwhale.git
|
|
||||||
cd funkwhale
|
|
||||||
|
|
||||||
|
|
||||||
A note about branches
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
Next release development occurs on the "develop" branch, and releases are made on the "master" branch. Therefor, when submitting Merge Requests, ensure you are merging on the develop branch.
|
|
||||||
|
|
||||||
|
|
||||||
Working with docker
|
|
||||||
^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
In developpement, we use the docker-compose file named ``dev.yml``, and this is why all our docker-compose commands will look like this::
|
|
||||||
|
|
||||||
docker-compose -f dev.yml logs
|
|
||||||
|
|
||||||
If you do not want to add the ``-f dev.yml`` snippet everytime, you can run this command before starting your work::
|
|
||||||
|
|
||||||
export COMPOSE_FILE=dev.yml
|
|
||||||
|
|
||||||
|
|
||||||
Building the containers
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
On your initial clone, or if there have been some changes in the
|
|
||||||
app dependencies, you will have to rebuild your containers. This is done
|
|
||||||
via the following command::
|
|
||||||
|
|
||||||
docker-compose -f dev.yml build
|
|
||||||
|
|
||||||
|
|
||||||
Creating your env file
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
We provide a working .env.dev configuration file that is suitable for
|
|
||||||
development. However, to enable customization on your machine, you should
|
|
||||||
also create a .env file that will hold your personal environment
|
|
||||||
variables (those will not be commited to the project).
|
|
||||||
|
|
||||||
Create it like this::
|
|
||||||
|
|
||||||
touch .env
|
|
||||||
|
|
||||||
|
|
||||||
Database management
|
|
||||||
^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
To setup funkwhale's database schema, run this::
|
|
||||||
|
|
||||||
docker-compose -f dev.yml run --rm api python manage.py migrate
|
|
||||||
|
|
||||||
This will create all the tables needed for the API to run proprely.
|
|
||||||
You will also need to run this whenever changes are made on the database
|
|
||||||
schema.
|
|
||||||
|
|
||||||
It is safe to run this command multiple times, so you can run it whenever
|
|
||||||
you fetch develop.
|
|
||||||
|
|
||||||
|
|
||||||
Development data
|
|
||||||
^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
You'll need at least an admin user and some artists/tracks/albums to work
|
|
||||||
locally.
|
|
||||||
|
|
||||||
Create an admin user with the following command::
|
|
||||||
|
|
||||||
docker-compose -f dev.yml run --rm api python manage.py createsuperuser
|
|
||||||
|
|
||||||
Injecting fake data is done by running the fllowing script::
|
|
||||||
|
|
||||||
artists=25
|
|
||||||
command="from funkwhale_api.music import fake_data; fake_data.create_data($artists)"
|
|
||||||
echo $command | docker-compose -f dev.yml run --rm api python manage.py shell -i python
|
|
||||||
|
|
||||||
The previous command will create 25 artists with random albums, tracks
|
|
||||||
and metadata.
|
|
||||||
|
|
||||||
|
|
||||||
Launch all services
|
|
||||||
^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
Then you can run everything with::
|
|
||||||
|
|
||||||
docker-compose -f dev.yml up
|
|
||||||
|
|
||||||
This will launch all services, and output the logs in your current terminal window.
|
|
||||||
If you prefer to launch them in the background instead, use the ``-d`` flag, and access the logs when you need it via ``docker-compose -f dev.yml logs --tail=50 --follow``.
|
|
||||||
|
|
||||||
Once everything is up, you can access the various funkwhale's components:
|
|
||||||
|
|
||||||
- The Vue webapp, on http://localhost:8080
|
|
||||||
- The API, on http://localhost:8080/api/v1/
|
|
||||||
- The django admin, on http://localhost:8080/api/admin/
|
|
||||||
|
|
||||||
|
|
||||||
Running API tests
|
|
||||||
^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
To run the pytest test suite, use the following command::
|
|
||||||
|
|
||||||
docker-compose -f dev.yml run --rm api pytest
|
|
||||||
|
|
||||||
This is regular pytest, so you can use any arguments/options that pytest usually accept::
|
|
||||||
|
|
||||||
# get some help
|
|
||||||
docker-compose -f dev.yml run --rm api pytest -h
|
|
||||||
# Stop on first failure
|
|
||||||
docker-compose -f dev.yml run --rm api pytest -x
|
|
||||||
# Run a specific test file
|
|
||||||
docker-compose -f dev.yml run --rm api pytest tests/test_acoustid.py
|
|
||||||
|
|
||||||
|
|
||||||
Running front-end tests
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
To run the front-end test suite, use the following command::
|
|
||||||
|
|
||||||
docker-compose -f dev.yml run --rm front yarn run unit
|
|
||||||
|
|
||||||
We also support a "watch and test" mode were we continually relaunch
|
|
||||||
tests when changes are recorded on the file system::
|
|
||||||
|
|
||||||
docker-compose -f dev.yml run --rm front yarn run unit-watch
|
|
||||||
|
|
||||||
The latter is especially useful when you are debugging failing tests.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
The front-end test suite coverage is still pretty low
|
|
||||||
|
|
||||||
|
|
||||||
Stopping everything
|
|
||||||
^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
Once you're down with your work, you can stop running containers, if any, with::
|
|
||||||
|
|
||||||
docker-compose -f dev.yml stop
|
|
||||||
|
|
||||||
|
|
||||||
Removing everything
|
|
||||||
^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
If you want to wipe your development environment completely (e.g. if you want to start over from scratch), just run::
|
|
||||||
|
|
||||||
docker-compose -f dev.yml down -v
|
|
||||||
|
|
||||||
This will wipe your containers and data, so please be careful before running it.
|
|
||||||
|
|
||||||
You can keep your data by removing the ``-v`` flag.
|
|
||||||
|
|
||||||
|
|
||||||
Typical workflow for a merge request
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
0. Fork the project if you did not already or if you do not have access to the main repository
|
|
||||||
1. Checkout the development branch and pull most recent changes: ``git checkout develop && git pull``
|
|
||||||
2. Create a dedicated branch for your work ``42-awesome-fix``. It is good practice to prefix your branch name with the ID of the issue you are solving.
|
|
||||||
3. Work on your stuff
|
|
||||||
4. Commit small, atomic changes to make it easier to review your contribution
|
|
||||||
5. Add a changelog fragment to summarize your changes: ``echo "Implemented awesome stuff (#42)" > changes/changelog.d/42.feature"``
|
|
||||||
6. Push your branch
|
|
||||||
7. Create your merge request
|
|
||||||
8. Take a step back and enjoy, we're really grateful you did all of this and took the time to contribute!
|
|
||||||
|
|
||||||
|
|
||||||
Internationalization
|
|
||||||
--------------------
|
|
||||||
|
|
||||||
When working on the front-end, any end-user string should be translated
|
|
||||||
using either ``<i18next path="yourstring">`` or the ``$t('yourstring')``
|
|
||||||
function.
|
|
||||||
|
|
||||||
Extraction is done by calling ``yarn run i18n-extract``, which
|
|
||||||
will pull all the strings from source files and put them in a PO file.
|
|
||||||
|
|
||||||
|
|
||||||
Working with federation locally
|
|
||||||
-------------------------------
|
|
||||||
|
|
||||||
To achieve that, you'll need:
|
|
||||||
|
|
||||||
1. to update your dns resolver to resolve all your .dev hostnames locally
|
|
||||||
2. a reverse proxy (such as traefik) to catch those .dev requests and
|
|
||||||
and with https certificate
|
|
||||||
3. two instances (or more) running locally, following the regular dev setup
|
|
||||||
|
|
||||||
Resolve .dev names locally
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
If you use dnsmasq, this is as simple as doing::
|
|
||||||
|
|
||||||
echo "address=/test/172.17.0.1" | sudo tee /etc/dnsmasq.d/test.conf
|
|
||||||
sudo systemctl restart dnsmasq
|
|
||||||
|
|
||||||
If you use NetworkManager with dnsmasq integration, use this instead::
|
|
||||||
|
|
||||||
echo "address=/test/172.17.0.1" | sudo tee /etc/NetworkManager/dnsmasq.d/test.conf
|
|
||||||
sudo systemctl restart NetworkManager
|
|
||||||
|
|
||||||
Add wildcard certificate to the trusted certificates
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
Simply copy bundled certificates::
|
|
||||||
|
|
||||||
sudo cp docker/ssl/test.crt /usr/local/share/ca-certificates/
|
|
||||||
sudo update-ca-certificates
|
|
||||||
|
|
||||||
This certificate is a wildcard for ``*.funkwhale.test``
|
|
||||||
|
|
||||||
Run a reverse proxy for your instances
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
|
|
||||||
Create docker network
|
|
||||||
^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
Create the federation network::
|
|
||||||
|
|
||||||
docker network create federation
|
|
||||||
|
|
||||||
Launch everything
|
|
||||||
^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
Launch the traefik proxy::
|
|
||||||
|
|
||||||
docker-compose -f docker/traefik.yml up -d
|
|
||||||
|
|
||||||
Then, in separate terminals, you can setup as many different instances as you
|
|
||||||
need::
|
|
||||||
|
|
||||||
export COMPOSE_PROJECT_NAME=node2
|
|
||||||
docker-compose -f dev.yml run --rm api python manage.py migrate
|
|
||||||
docker-compose -f dev.yml run --rm api python manage.py createsuperuser
|
|
||||||
docker-compose -f dev.yml up nginx api front nginx api celeryworker
|
|
||||||
|
|
||||||
Note that by default, if you don't export the COMPOSE_PROJECT_NAME,
|
|
||||||
we will default to node1 as the name of your instance.
|
|
||||||
|
|
||||||
Assuming your project name is ``node1``, your server will be reachable
|
|
||||||
at ``https://node1.funkwhale.test/``. Not that you'll have to trust
|
|
||||||
the SSL Certificate as it's self signed.
|
|
||||||
|
|
||||||
When working on federation with traefik, ensure you have this in your ``env``::
|
|
||||||
|
|
||||||
# This will ensure we don't bind any port on the host, and thus enable
|
|
||||||
# multiple instances of funkwhale to be spawned concurrently.
|
|
||||||
WEBPACK_DEVSERVER_PORT_BINDING=
|
|
||||||
# This disable certificate verification
|
|
||||||
EXTERNAL_REQUESTS_VERIFY_SSL=false
|
|
||||||
# this ensure you don't have incorrect urls pointing to http resources
|
|
||||||
FUNKWHALE_PROTOCOL=https
|
|
||||||
|
|
|
@ -27,12 +27,19 @@ def factories_autodiscover():
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
@pytest.fixture(autouse=True)
|
||||||
def cache():
|
def cache():
|
||||||
|
"""
|
||||||
|
Returns a django Cache instance for cache-related operations
|
||||||
|
"""
|
||||||
yield django_cache
|
yield django_cache
|
||||||
django_cache.clear()
|
django_cache.clear()
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def factories(db):
|
def factories(db):
|
||||||
|
"""
|
||||||
|
Returns a dictionnary containing all registered factories with keys such as
|
||||||
|
users.User or music.Track
|
||||||
|
"""
|
||||||
from funkwhale_api import factories
|
from funkwhale_api import factories
|
||||||
for v in factories.registry.values():
|
for v in factories.registry.values():
|
||||||
try:
|
try:
|
||||||
|
@ -45,6 +52,10 @@ def factories(db):
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def nodb_factories():
|
def nodb_factories():
|
||||||
|
"""
|
||||||
|
Returns a dictionnary containing all registered factories with a build strategy
|
||||||
|
that does not require access to the database
|
||||||
|
"""
|
||||||
from funkwhale_api import factories
|
from funkwhale_api import factories
|
||||||
for v in factories.registry.values():
|
for v in factories.registry.values():
|
||||||
try:
|
try:
|
||||||
|
@ -57,6 +68,9 @@ def nodb_factories():
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def preferences(db, cache):
|
def preferences(db, cache):
|
||||||
|
"""
|
||||||
|
return a dynamic_preferences manager for global_preferences
|
||||||
|
"""
|
||||||
manager = global_preferences_registry.manager()
|
manager = global_preferences_registry.manager()
|
||||||
manager.all()
|
manager.all()
|
||||||
yield manager
|
yield manager
|
||||||
|
@ -64,6 +78,10 @@ def preferences(db, cache):
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def tmpdir():
|
def tmpdir():
|
||||||
|
"""
|
||||||
|
Returns a temporary directory path where you can write things during your
|
||||||
|
test
|
||||||
|
"""
|
||||||
d = tempfile.mkdtemp()
|
d = tempfile.mkdtemp()
|
||||||
yield d
|
yield d
|
||||||
shutil.rmtree(d)
|
shutil.rmtree(d)
|
||||||
|
@ -71,11 +89,18 @@ def tmpdir():
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def tmpfile():
|
def tmpfile():
|
||||||
|
"""
|
||||||
|
Returns a temporary file where you can write things during your test
|
||||||
|
"""
|
||||||
yield tempfile.NamedTemporaryFile()
|
yield tempfile.NamedTemporaryFile()
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def logged_in_client(db, factories, client):
|
def logged_in_client(db, factories, client):
|
||||||
|
"""
|
||||||
|
Returns a logged-in, non-API client with an authenticated ``User``
|
||||||
|
stored in the ``user`` attribute
|
||||||
|
"""
|
||||||
user = factories['users.User']()
|
user = factories['users.User']()
|
||||||
assert client.login(username=user.username, password='test')
|
assert client.login(username=user.username, password='test')
|
||||||
setattr(client, 'user', user)
|
setattr(client, 'user', user)
|
||||||
|
@ -85,16 +110,24 @@ def logged_in_client(db, factories, client):
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def anonymous_user():
|
def anonymous_user():
|
||||||
|
"""Returns a AnonymousUser() instance"""
|
||||||
return AnonymousUser()
|
return AnonymousUser()
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def api_client(client):
|
def api_client(client):
|
||||||
|
"""
|
||||||
|
Return an API client without any authentication
|
||||||
|
"""
|
||||||
return APIClient()
|
return APIClient()
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def logged_in_api_client(db, factories, api_client):
|
def logged_in_api_client(db, factories, api_client):
|
||||||
|
"""
|
||||||
|
Return a logged-in API client with an authenticated ``User``
|
||||||
|
stored in the ``user`` attribute
|
||||||
|
"""
|
||||||
user = factories['users.User']()
|
user = factories['users.User']()
|
||||||
assert api_client.login(username=user.username, password='test')
|
assert api_client.login(username=user.username, password='test')
|
||||||
setattr(api_client, 'user', user)
|
setattr(api_client, 'user', user)
|
||||||
|
@ -104,6 +137,10 @@ def logged_in_api_client(db, factories, api_client):
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def superuser_api_client(db, factories, api_client):
|
def superuser_api_client(db, factories, api_client):
|
||||||
|
"""
|
||||||
|
Return a logged-in API client with an authenticated superuser
|
||||||
|
stored in the ``user`` attribute
|
||||||
|
"""
|
||||||
user = factories['users.SuperUser']()
|
user = factories['users.SuperUser']()
|
||||||
assert api_client.login(username=user.username, password='test')
|
assert api_client.login(username=user.username, password='test')
|
||||||
setattr(api_client, 'user', user)
|
setattr(api_client, 'user', user)
|
||||||
|
@ -113,6 +150,10 @@ def superuser_api_client(db, factories, api_client):
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def superuser_client(db, factories, client):
|
def superuser_client(db, factories, client):
|
||||||
|
"""
|
||||||
|
Return a logged-in, non-API client with an authenticated ``User``
|
||||||
|
stored in the ``user`` attribute
|
||||||
|
"""
|
||||||
user = factories['users.SuperUser']()
|
user = factories['users.SuperUser']()
|
||||||
assert client.login(username=user.username, password='test')
|
assert client.login(username=user.username, password='test')
|
||||||
setattr(client, 'user', user)
|
setattr(client, 'user', user)
|
||||||
|
@ -122,11 +163,17 @@ def superuser_client(db, factories, client):
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def api_request():
|
def api_request():
|
||||||
|
"""
|
||||||
|
Returns a dummy API request object you can pass to API views
|
||||||
|
"""
|
||||||
return APIRequestFactory()
|
return APIRequestFactory()
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def fake_request():
|
def fake_request():
|
||||||
|
"""
|
||||||
|
Returns a dummy, non-API request object you can pass to regular views
|
||||||
|
"""
|
||||||
return client.RequestFactory()
|
return client.RequestFactory()
|
||||||
|
|
||||||
|
|
||||||
|
@ -140,16 +187,6 @@ def activity_registry():
|
||||||
record.registry[key] = value
|
record.registry[key] = value
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def activity_registry():
|
|
||||||
r = record.registry
|
|
||||||
state = list(record.registry.items())
|
|
||||||
yield record.registry
|
|
||||||
record.registry.clear()
|
|
||||||
for key, value in state:
|
|
||||||
record.registry[key] = value
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def activity_muted(activity_registry, mocker):
|
def activity_muted(activity_registry, mocker):
|
||||||
yield mocker.patch.object(record, 'send')
|
yield mocker.patch.object(record, 'send')
|
||||||
|
@ -157,6 +194,9 @@ def activity_muted(activity_registry, mocker):
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
@pytest.fixture(autouse=True)
|
||||||
def media_root(settings):
|
def media_root(settings):
|
||||||
|
"""
|
||||||
|
Sets settings.MEDIA_ROOT to a temporary path and returns this path
|
||||||
|
"""
|
||||||
tmp_dir = tempfile.mkdtemp()
|
tmp_dir = tempfile.mkdtemp()
|
||||||
settings.MEDIA_ROOT = tmp_dir
|
settings.MEDIA_ROOT = tmp_dir
|
||||||
yield settings.MEDIA_ROOT
|
yield settings.MEDIA_ROOT
|
||||||
|
@ -165,12 +205,19 @@ def media_root(settings):
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def r_mock():
|
def r_mock():
|
||||||
|
"""
|
||||||
|
Returns a requests_mock.mock() object you can use to mock HTTP calls made
|
||||||
|
using python-requests
|
||||||
|
"""
|
||||||
with requests_mock.mock() as m:
|
with requests_mock.mock() as m:
|
||||||
yield m
|
yield m
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def authenticated_actor(factories, mocker):
|
def authenticated_actor(factories, mocker):
|
||||||
|
"""
|
||||||
|
Returns an authenticated ActivityPub actor
|
||||||
|
"""
|
||||||
actor = factories['federation.Actor']()
|
actor = factories['federation.Actor']()
|
||||||
mocker.patch(
|
mocker.patch(
|
||||||
'funkwhale_api.federation.authentication.SignatureAuthentication.authenticate_actor',
|
'funkwhale_api.federation.authentication.SignatureAuthentication.authenticate_actor',
|
||||||
|
|
File diff suppressed because one or more lines are too long
|
@ -0,0 +1 @@
|
||||||
|
.. include:: ../CONTRIBUTING
|
|
@ -19,6 +19,7 @@ Funkwhale is a self-hosted, modern free and open-source music server, heavily in
|
||||||
api
|
api
|
||||||
upgrading
|
upgrading
|
||||||
third-party
|
third-party
|
||||||
|
contributing
|
||||||
changelog
|
changelog
|
||||||
|
|
||||||
Indices and tables
|
Indices and tables
|
||||||
|
|
Loading…
Reference in New Issue