Rewrite architecture doc
This commit is contained in:
Ciarán Ainsworth
parent
5dc7a33a2f
commit
337942a97c
|
|
@ -0,0 +1 @@
|
|||
Rewrote the architecture file (#1908)
|
||||
|
|
@ -0,0 +1,107 @@
|
|||
# Project architecture
|
||||
|
||||
Funkwhale is made up of several components. Understanding these components and what they do is important when contributing to Funkwhale's codebase. In this article, we'll break down each part of Funkwhale's architecture to help you understand what each component does.
|
||||
|
||||
Below is a diagram of Funkwhale's project setup.
|
||||
|
||||
```{mermaid}
|
||||
flowchart TD
|
||||
accTitle: Funkwhale data flow diagram
|
||||
accDescr: A diagram showing the components of the Funkwhale app and how data flows through each component.
|
||||
subgraph Entrypoints
|
||||
user[User] --> frontend[Funkwhale web app]
|
||||
user --> ffa[Funkwhale for Android]
|
||||
user --> subsonic[Subsonic app]
|
||||
frontend --> proxy[Nginx/Apache reverse proxy]
|
||||
ffa --> proxy
|
||||
subsonic --> proxy
|
||||
end
|
||||
subgraph Funkwhale backend
|
||||
proxy --> api[Django API server]
|
||||
api --> db[PostgreSQL database]
|
||||
api --> redis[Redis cache and message queue]
|
||||
beat[Celery beat task scheduler] --> redis
|
||||
redis <--> celery[Celery worker]
|
||||
celery --> db
|
||||
end
|
||||
```
|
||||
|
||||
Select a link below to see information about each component.
|
||||
|
||||
```{contents}
|
||||
:local:
|
||||
```
|
||||
|
||||
## Entrypoints
|
||||
|
||||
Users can access Funkwhale using a variety of entrypoints. They can make use of a Funkwhale application, a Subsonic-compatible application, or by calling the API directly. Each entrypoint interacts with the Funkwhale backend in the same way.
|
||||
|
||||
### Funkwhale web app
|
||||
|
||||
The Funkwhale web app is a {abbr}`SPA (Single Page Application)` written in [Vue.js](https://vuejs.org) and [Typescript](https://typescriptlang.org). This is the application most people associate with Funkwhale. Server admins usually run an instance of the web app alongside their Funkwhale {term}`pod`, but you can also connect a standalone web app to another pod.
|
||||
|
||||
The Funkwhale web app interacts with the Funkwhale API to fetch and update data. Using a service worker, the web app caches important information for offline use.
|
||||
|
||||
### Funkwhale for Android
|
||||
|
||||
Funkwhale for Android is the Funkwhale collective's official Android app written in [Kotlin](https://kotlinlang.org/). It interacts with the Funkwhale API to fetch and update data and stores information for offline playback.
|
||||
|
||||
### Subsonic app
|
||||
|
||||
Funkwhale supports a limited subset of the [Subsonic API](http://www.subsonic.org/pages/api.jsp) to support existing Subsonic apps. These apps can request data stored on a Funkwhale server by calling these endpoints.
|
||||
|
||||
### Nginx/Apache reverse proxy
|
||||
|
||||
The reverse proxy acts as a layer between a Funkwhale pod and the open internet. It enhances the pod's security and provides additional options to help increase performance.
|
||||
|
||||
When a user tries to communicate with a Funkwhale pod, the reverse proxy:
|
||||
|
||||
1. Handles the HTTP/HTTPS requests and proxies them to the Funkwhale API server
|
||||
2. Serves requested static files, such as audio files and stylesheets
|
||||
|
||||
## Backend
|
||||
|
||||
The Funkwhale backend is made up of a few components which are responsible for:
|
||||
|
||||
1. Communicating with the user's entrypoint and actioning requests
|
||||
2. Maintaining data consistency
|
||||
3. Communicating with other Funkwhale pods (if federation is enabled)
|
||||
|
||||
### Django API server
|
||||
|
||||
The Funkwhale API is a [REST API](https://developer.mozilla.org/en-US/docs/Glossary/REST) written in [Python](https://www.python.org/) using the [Django REST framework](https://www.django-rest-framework.org/). It is the central piece of the project and houses the application's logic.
|
||||
|
||||
The Funkwhale API is responsible for:
|
||||
|
||||
1. Fetching requested data from the cache/database and returning it to the requester in a meaningful way
|
||||
2. Processing incoming data and writing it to the database in a meaningful way
|
||||
3. Delegating long-running tasks to workers to reduce load
|
||||
|
||||
### PostgreSQL database
|
||||
|
||||
Funkwhale uses a [PostgreSQL database](https://www.postgresql.org/) to store data. All information that is served by and sent to the Funkwhale API is stored in this database.
|
||||
|
||||
The Funkwhale database makes heavy use of [indexes](https://www.postgresql.org/docs/current/indexes.html) for enhanced performance.
|
||||
|
||||
### Redis cache and message queue
|
||||
|
||||
Funkwhale uses [Redis](https://redis.io/) to cache information from the database and to store a queue of messages to send. We use this cache to avoid locking database resources and to speed up requests.
|
||||
|
||||
### Celery worker
|
||||
|
||||
Funkwhale has to handle a lot of tasks that take longer than the average HTTP request/response cycle. To ensure these tasks complete and don't impact the API's performance, they are offloaded to a [Celery](https://docs.celeryq.dev/en/stable/userguide/workers.html) task worker. The worker then works through all the tasks in its queue while the API handles real-time responses.
|
||||
|
||||
Some common tasks the Celery worker handles are:
|
||||
|
||||
- Importing uploaded music to the database
|
||||
- Handling [ActivityPub](https://www.w3.org/TR/activitypub/) messages from other {term}`Fediverse` servers
|
||||
- Scanning new content on remote pods
|
||||
|
||||
### Celery beat task scheduler
|
||||
|
||||
In addition to handling tasks from the API, the Celery worker also needs to handle some recurring tasks. To manage these, we implement a [Celery beat]() scheduler. The scheduler is responsible for triggering tasks on a schedule and adding messages to the [queue](#redis-cache-and-message-queue) so the worker can work through them.
|
||||
|
||||
Some common recurring tasks are:
|
||||
|
||||
- Clearing the cache
|
||||
- Refreshing content metadata
|
||||
|
|
@ -1,118 +0,0 @@
|
|||
Architecture
|
||||
============
|
||||
|
||||
Funkwhale is made of several components, each of them fulfilling a specific mission:
|
||||
|
||||
.. graphviz::
|
||||
|
||||
digraph {
|
||||
node [shape=record];
|
||||
rankdir=TB
|
||||
concentrate=true
|
||||
user [group="frontend" label="User" fontsize="9"]
|
||||
webui [group="frontend" label="Web interface (VueJS SPA)" fontsize="9"]
|
||||
subapps [group="frontend" label="Subsonic-compatible apps (DSub, Clementine)" fontsize="9"]
|
||||
proxy [label="Reverse proxy (Nginx/Apache)" fontsize="9"]
|
||||
api [label="API Server (Django)" fontsize="9"]
|
||||
db [label="Database (PostgreSQL)" fontsize="9"]
|
||||
cache [label="Cache and message queue (Redis)" fontsize="9"]
|
||||
worker [label="Worker (Celery)" fontsize="9"]
|
||||
scheduler [label="Task scheduler (Celery Beat)" fontsize="9"]
|
||||
|
||||
user -> subapps -> proxy
|
||||
user -> webui -> proxy
|
||||
cache -> worker
|
||||
proxy -> api
|
||||
api -> cache
|
||||
api -> db
|
||||
scheduler -> cache
|
||||
worker -> cache
|
||||
worker -> db
|
||||
}
|
||||
|
||||
This graph may looks a bit scary, so we'll detail the role of each component below.
|
||||
|
||||
The user
|
||||
--------
|
||||
|
||||
Funkwhale users can interact with your instance using:
|
||||
|
||||
- The official web interface
|
||||
- Third-party apps
|
||||
|
||||
The web interface
|
||||
-----------------
|
||||
|
||||
This refers to Funkwhale's built-in web interface, which is a Single Page application
|
||||
written in Vue JS. This application will interact with Funkwhale's API to retrieve
|
||||
or persist data.
|
||||
|
||||
Third-party apps
|
||||
----------------
|
||||
|
||||
Since Funkwhale implements a subset of the Subsonic API, it's compatible with existing `apps <https://funkwhale.audio/en_US/apps>`_ such
|
||||
as DSub, Ultrasonic or Clementine that support this API. Those apps can be used as a replacement
|
||||
or in conjunction of the web interface, but the underlying data is the same.
|
||||
|
||||
The reverse proxy
|
||||
-----------------
|
||||
|
||||
Funkwhale's API server should never be exposed directly to the internet, as we require
|
||||
a reverse proxy (Apache or Nginx) for performance and security reasons. The reverse proxy
|
||||
will receive client HTTP or HTTPS requests, and:
|
||||
|
||||
- Proxy them to the API server
|
||||
- Serve requested static files (audio files, stylesheets, javascript, fonts...)
|
||||
|
||||
The API server
|
||||
--------------
|
||||
|
||||
Funkwhale's API server is the central piece of the project. This component is responsible
|
||||
for answering and processing user requests, manipulate data from the database, send long-running
|
||||
tasks to workers, etc.
|
||||
|
||||
It's a Python/Django application.
|
||||
|
||||
The database
|
||||
------------
|
||||
|
||||
Most of the data such as user accounts, favorites, music metadata or playlist is stored
|
||||
in a PostgreSQL database.
|
||||
|
||||
The cache/message queue
|
||||
-----------------------
|
||||
|
||||
Fetching data from the database is sometimes slow or resource hungry. To reduce
|
||||
the load, Redis act as a cache for data that is considerably faster than a database.
|
||||
|
||||
It is also a message queue that will deliver tasks to the worker.
|
||||
|
||||
The worker
|
||||
----------
|
||||
|
||||
Some operations are too long to live in the HTTP request/response cycle. Typically,
|
||||
importing a bunch of uploaded tracks could take a minute or two.
|
||||
|
||||
To keep the API response time as fast as possible, we offload long-running tasks
|
||||
to a background process, also known as the Celery worker.
|
||||
|
||||
Typical tasks include:
|
||||
|
||||
- Handling music imports
|
||||
- Handling federation/ActivityPub messages
|
||||
- Scanning other instances libraries
|
||||
|
||||
This worker is also able to retry failed tasks, or spawn automatically
|
||||
more process when the number of received tasks increase.
|
||||
|
||||
The scheduler
|
||||
-------------
|
||||
|
||||
Some long-running tasks are not triggered by user or external input, but on a recurring
|
||||
basis instead. The scheduler is responsible for triggering those tasks and put the corresponding
|
||||
messages in the message queue so the worker can process them.
|
||||
|
||||
Recurring tasks include:
|
||||
|
||||
- Cache cleaning
|
||||
- Music metadata refreshing
|
||||
|
|
@ -7,7 +7,6 @@ how Funkwhale works and how to build apps that integrate with Funkwhale's ecosys
|
|||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
architecture
|
||||
../api
|
||||
./authentication
|
||||
./plugins
|
||||
|
|
|
|||
|
|
@ -71,6 +71,7 @@ hidden: true
|
|||
---
|
||||
|
||||
developer_documentation/index
|
||||
developer_documentation/architecture
|
||||
developer_documentation/setup/index
|
||||
developer_documentation/contribute/index
|
||||
developer_documentation/workflows/index
|
||||
|
|
|
|||
Loading…
Reference in New Issue