ff9fe2c1be
Remove record equality test
2023-08-17 13:55:27 -07:00
7f37c8ee5e
Retire now-unused HTTP transport configuration for Datadog metric reporter
2023-08-17 16:53:53 -04:00
ed0a723fef
Include underlying exceptions when logging failures to write exit files
2023-08-17 12:32:45 -04:00
5c31ef43c9
Send an HTTP/440 response instead of an HTTP/502 if an upstream provider rejects a "send verification code" request
2023-08-17 12:15:00 -04:00
43fd8518c0
Add missing `java.util.Base64` import to ProfileController
2023-08-16 14:02:53 -07:00
19a08f01e8
Write certain profile data as bytes instead of strings to dynamo and represent those fields as byte arrays on `VersionedProfile`
2023-08-16 13:45:16 -07:00
ccd860207b
Make `MessagesManager#clear` asynchronous
2023-08-15 14:08:16 -04:00
2c835b5c51
Make message deletion from DynamoDB asynchronous
2023-08-15 14:08:16 -04:00
5caa951c61
Make `MessagesCache#clear` methods asynchronous
2023-08-15 14:08:16 -04:00
4d8c4d6693
Also delete APNs VOIP tokens when clearing APNs tokens
2023-08-15 14:08:00 -04:00
a9d0574ea8
Remove most `@Timed` annotations
2023-08-15 14:06:31 -04:00
ed6a2c55eb
adjust lettuce metric denial for post-transform name
2023-08-11 09:43:41 -05:00
b6ee074149
fix captcha shortening url path resolution
2023-08-10 16:01:56 -05:00
f6b3500e92
remove most high cardinality lettuce metrics
2023-08-10 16:01:16 -05:00
a71dc48b9b
Prepare to read profile data stored as byte arrays
2023-08-10 14:00:35 -07:00
bc5eed48c3
Add authentication interceptor to profile gRPC service
2023-08-10 13:59:46 -07:00
2ecf3cb303
Revert "Don't immediately require PNI-associated keys for "atomic" device linking"
...
This reverts commit 4ec97cf006
2023-08-10 16:59:35 -04:00
bed33d042a
Revert "Require PNI-associated keys if the target account has a PNI identity key"
...
This reverts commit 1dde612855
2023-08-10 16:59:35 -04:00
3ac7aba6b2
Add a captcha short-code expander
2023-08-09 12:41:31 -05:00
1dde612855
Require PNI-associated keys if the target account has a PNI identity key
2023-08-09 12:10:56 -04:00
4ec97cf006
Don't immediately require PNI-associated keys for "atomic" device linking
2023-08-09 12:10:56 -04:00
d51c6fd2f8
Convert `Device.Capabilities` to a record
2023-08-08 15:38:37 -04:00
d868e3075c
Retire fully-adopted device capabilities
2023-08-08 15:38:37 -04:00
ae61ee5486
Retire `AnalyzeDeviceCapabilitiesCommand`
2023-08-08 15:38:37 -04:00
58fd9ddb27
Count profile data that cannot be parsed as base64
2023-08-08 10:54:25 -07:00
a953cb33b7
Define ProfileController protobufs and setProfile endpoint
2023-08-08 10:53:11 -07:00
95b90e7c5a
Add a preliminary gRPC service for dealing with calling credentials
2023-08-08 12:46:55 -04:00
6a3ecb2881
Convert `TurnToken` to a record
2023-08-08 12:46:55 -04:00
6cf4241283
Add a reactive method for checking rate limits by UUID
2023-08-08 12:46:55 -04:00
42141e51a1
Use ACIs instead of E164s for TURN URI overrides
2023-08-08 12:46:55 -04:00
b01945ff50
Clarify parameterized tests by modifying prototype request objects; remove spurious warning suppressions
2023-08-08 10:33:29 -04:00
a131f2116f
Retire verification code storage machinery
2023-08-04 17:26:55 -04:00
625637b888
Stop checking for stored verification codes when linking devices
2023-08-04 17:26:55 -04:00
c873f62025
Produce verification tokens instead of stored verification codes for linking devices
2023-08-04 16:04:47 -04:00
43d91e5bd6
Convert `VerificationCode` to a record
2023-08-04 16:04:47 -04:00
5c4c729703
Disallow reuse of device verification tokens
2023-08-04 13:40:37 -05:00
308da3343d
Accept signed tokens in addition to randomly-generated codes for authorizing device linking
2023-08-04 13:40:37 -05:00
48c7572dd5
Add CommandStopListener
2023-08-04 13:29:35 -05:00
69ea9b0296
Add a request counter tagged by client version
2023-08-04 12:16:48 -04:00
969c6884c0
Add a command for analyzing device capabilities
2023-08-04 12:14:08 -04:00
fcf311aab3
Retire the `PendingAccounts` table
2023-08-04 12:13:57 -04:00
888879dfb2
Estimate message byte limit exceeded error count
2023-08-04 11:10:58 -05:00
f53743d287
Add configuration for Datadog UDP transport
2023-08-02 13:54:15 -05:00
2d132128e1
Switched to a composed request object model for anonymous keys gRPC operations
2023-07-28 14:20:24 -05:00
6e5ffbe7b5
Restore `aci` field to `BatchIdentityCheckRequest`
2023-07-28 14:16:48 -05:00
baf98accd0
acquire lock before checking message listeners in MessagesCache
2023-07-26 16:45:53 -04:00
901c950ee6
Add metrics to keyspace-notifier executor
2023-07-26 16:45:53 -04:00
50ac7f9dc2
adjust messageDeletionAsyncExecutor core pool size
2023-07-26 16:45:41 -04:00
b691b8d37d
Log successful client version refreshes
2023-07-26 16:41:54 -04:00
4ead8527c8
Use `ClientReleasesManager` when deciding whether to add client version tags
2023-07-26 16:41:54 -04:00
6f4801fd6f
Add a manager class for checking "liveness" of client versions
2023-07-26 16:41:54 -04:00
10689843b0
Add a repository for client release information
2023-07-26 16:41:54 -04:00
60cc0c482e
Add `@Produces` to `PUT /v1/accounts/apn`
2023-07-26 16:35:23 -04:00
e1a5105c28
Revert "Restore max concurrency when migrating pre-keys"
...
This reverts commit ed8a1ed579
2023-07-26 12:56:35 -04:00
ed8a1ed579
Restore max concurrency when migrating pre-keys
2023-07-26 12:34:32 -04:00
c3fd2e2284
Retry key storage attempts when migrating signed pre-keys
2023-07-26 12:34:32 -04:00
872ef5d0a0
Add environment variable to toggle tcp appender
2023-07-24 13:13:13 -05:00
b44599cd59
Remove unused jedis library
2023-07-24 10:54:34 -04:00
7a5dcc700e
Add support for AuthCredentialAsPni with pniAsServiceId=true
...
Update to libsignal 0.30.0, and add a new query param to
/v1/certificate/auth/group, "pniAsServiceId=true", that uses the new
encoding of PNIs in zkgroup rather than encoding PNIs the same way as
ACIs, as we have been doing.
Also includes all the adjustments for the libsignal 0.30.0 update.
2023-07-24 10:53:59 -04:00
705fb93e45
Add v4 attachment controller
...
Add AttachmentControllerV4 which can be configured to generate upload
forms for a TUS based CDN
2023-07-21 12:09:45 -05:00
9df923d916
Update keys gRPC endpoint to use service identifiers
2023-07-21 13:03:01 -04:00
dc1cb9093a
Remove unused code
2023-07-21 11:08:32 -05:00
e32043ae79
Remove outdated documentation
2023-07-21 10:24:17 -04:00
abb32bd919
Introduce "service identifiers"
2023-07-21 09:34:10 -04:00
cf92007f66
Moving Account serialization logic to storage-specific classes
2023-07-20 14:28:07 -07:00
f5c57e5741
Make `ContestedOptimisticLockException` extend `NoStackTraceRuntimeException`
2023-07-20 11:15:08 -04:00
5627209fdd
Add a gRPC service for working with pre-keys
2023-07-20 11:10:26 -04:00
0188d314ce
minor username api refinements
2023-07-19 15:12:47 -07:00
67343f6bdc
accept encrypted username with confirm-username-hash requests
2023-07-19 10:54:11 -07:00
ade2e9c6cf
Define asynchronous `ProfilesManager` operations
2023-07-19 10:43:58 -07:00
352e1b2249
test classes moved to same packages with components they test
2023-07-17 13:34:58 -07:00
b8d8d349f4
Control inbound message byte limits with a dynamic configuration flag
2023-07-14 16:25:33 -04:00
e87468fbe0
Add a rate limit for inbound message bytes for a given account
2023-07-14 16:25:33 -04:00
e38a713ccc
Support sub-millisecond permit regeneration durations in rate limiters
2023-07-14 16:25:33 -04:00
82ed783a2d
Introduce async account updaters
2023-07-14 16:25:19 -04:00
d17c7aaba6
Add support for clearing accounts from Redis asynchronously
2023-07-14 16:25:19 -04:00
41f61c66a3
Add public methods for fetching accounts asynchronously
2023-07-13 13:53:29 -04:00
1b7a20619e
Add tools for testing asynchronous Redis operations
2023-07-13 13:53:29 -04:00
7d19e58953
Add parallel pathways for getting accounts asyncronously to `Accounts`
2023-07-13 13:53:29 -04:00
1605676509
Store signed EC pre-keys in a dedicated table when setting signed pre-keys individually
2023-07-12 14:58:10 -04:00
a0d6146ff5
Make key deletion operations asynchronous
2023-07-12 14:58:10 -04:00
f709b00be3
Make `KeysManager` storage/retrieval operations asynchronous
2023-07-12 14:58:10 -04:00
5847300290
Revert "Allow use of the token returned with spam challenges as auth for the challenge verification request"
2023-07-12 11:45:02 -07:00
9aaac0eefd
don't require all devices to support PNI for PNIHW
2023-07-12 10:14:16 -07:00
6db97f5541
Standardize client tag version handling; add client version tags to delivery latency metrics
2023-07-11 13:35:29 -04:00
adf6c751ee
Use an explicit-allow model for tagging client versions in metrics
2023-07-11 13:35:29 -04:00
c315b34395
Update formatting in `UserAgentTagUtil`
2023-07-11 13:35:29 -04:00
f592201e4c
Limit attachment controller tags to UA platform (instead of platform and version)
2023-07-11 13:35:29 -04:00
8bf5ee45ed
Filter out `command` tags from Lettuce metrics and prepend a "chat." prefix to Lettuce metric names
2023-07-11 13:35:03 -04:00
25f759dd07
Drop `ActiveUserTally`
2023-07-11 13:34:36 -04:00
e5f4c17148
update openapi docs for several endpoints, notably those with PQXDH changes
...
Co-authored-by: Katherine Yen <katherine@signal.org>
2023-07-06 15:45:33 -07:00
098b177bd3
Allow use of the token returned with spam challenges as auth for the challenge verification request
2023-07-06 15:25:19 -07:00
ef1a8fc50f
Use PascalCase RPC names for the registration service
2023-07-06 17:12:37 -04:00
76f2e93a2c
Reduce concurrency limit for pre-key migration task
2023-07-06 16:45:03 -04:00
25ea1df299
Limit concurrency when writing signed EC pre-keys
2023-07-06 15:51:12 -04:00
5ced86af1d
Set `consistentRead=true` for registration recovery password lookup
...
This avoids a race condition (in integration test situations) where a lookup could return no results
2023-07-06 15:47:16 -04:00
62e02a49df
Log errors from single-shot account crawlers rather than printing them to stderr
2023-07-06 15:46:28 -04:00
540550d72a
Handle exceptions thrown when checking pre-key signatures
2023-07-06 15:46:11 -04:00
8cb83fb6e4
Switch to temporary registration endpoints to facilitate a change from snake_case to PascalCase
2023-07-06 15:46:00 -04:00
2c0fc8fe3e
Remove legacy methods from `RegistrationServiceClient`
2023-07-06 10:32:58 -04:00
08c7baafac
Remove legacy registration endpoints from `AccountController`
2023-07-06 10:32:58 -04:00
8edb450d73
Parallelize single-shot account crawlers
2023-07-06 10:15:16 -04:00
fedeef4da5
Add an optional parameter to require atomic account creation
...
By default, if a registration request has no optional fields for atomic
account creation set, the request will proceed non-atomically. If a
client sets the `atomic` field, now such a request would be rejected.
2023-07-05 11:24:11 -05:00
b593d49399
Control signed pre-key deletion via a dynamic configuration flag to facilitate migration
2023-07-05 12:17:17 -04:00
4a91fc3c3d
Set `daemon=true` for pubsub topology change event thread
2023-07-05 11:15:12 -05:00
bb9605d7c3
Use RedisClient#setDefaultTimeout for a non-clustered client
2023-07-05 11:09:28 -05:00
1049326a70
Turn on Lettuce latency metrics
2023-06-30 18:20:43 -04:00
457ecf145f
Add test for Redis timeouts
2023-06-30 12:55:37 -05:00
bdcd055aaf
Configure Redis timeouts using TimeoutOptions and RediURI
2023-06-30 12:55:37 -05:00
30ae2037e8
Correct order of constructor arguments for `KeysManager`
2023-06-30 12:05:16 -04:00
ce4fdbfb3c
Untangle metric names for `RepeatedUseSignedPreKeyStore` subclasses
2023-06-30 10:33:24 -04:00
2d154eb0cf
Add a command to copy signed pre-keys from `Account` records to their own table
2023-06-30 10:33:24 -04:00
a3e82dfae8
Add a temporary method for storing signed EC pre-keys if and only if another key has not already been stored
2023-06-30 10:33:24 -04:00
97a7469432
Measure signed EC pre-key agreement
2023-06-30 10:33:24 -04:00
1a1defb055
Store signed EC pre-keys in a dedicated table
2023-06-30 10:33:24 -04:00
93c78b6e40
Introduce `RepeatedUseECSignedPreKeyStore`
2023-06-30 10:33:24 -04:00
b852d6681d
FaultTolerantHttpClient: used managed ScheduledExecutorService for retries
2023-06-30 10:24:18 -04:00
8e48ac4ede
Add `messagesCache` and `clientPresenceManager` to managed command dependencies
2023-06-30 10:24:18 -04:00
859f646c55
Correct timestamp resolution to intended integer value
2023-06-29 16:05:59 -05:00
fb39b2edaf
Improve two `@Disabled` flaky tests
2023-06-29 14:56:41 -05:00
c93af9e31e
Remove `MessagePersister` from WhisperServerService environment
...
Persistence is now exclusively done by a separate command.
2023-06-28 14:17:49 -05:00
b81a0e99d4
Always have 0 `ApnPushNotificationScheduler` worker threads in front-end service
2023-06-28 14:17:23 -05:00
f8fefe2e5e
Remove `AccountCrawler` (and `doPeriodicWork`) from `WhisperServerService`
2023-06-28 14:16:07 -05:00
f26bc70b59
Add a basic, prototype authentication interceptor for gRPC services
2023-06-27 10:21:11 -04:00
b5fd131aba
Add an abstract base class for single-shot account crawls
2023-06-27 10:18:35 -04:00
06997e19e0
Add a method for iterating across all accounts
2023-06-27 10:18:35 -04:00
97710540c0
Use `Timer.Sample` throughout `Experiment`
2023-06-27 10:18:20 -04:00
c78c109577
Drop a disused endpoint for fetching the caller's own signed pre-key
2023-06-27 10:16:39 -04:00
8d995e456e
initial grpc service code in chat
2023-06-26 17:10:13 -07:00
cc3cab9c88
Add server time to remote config fetch
...
Enable clients to very roughly adjust some actions for clock skew by
providing current server time in the remote config fetch.
2023-06-21 17:11:35 -05:00
0122b410be
Include push notification urgency in push latency metrics
2023-06-21 15:10:26 -04:00
2ddd2b9476
Convert `PushRecord` to a `record` and make `PushType` non-optional
2023-06-21 15:10:26 -04:00
a768498250
Record general message delivery latency
2023-06-21 15:10:14 -04:00
a45aadae16
Cleaning up references to the legacy format from the rate limiters lua script
2023-06-21 15:09:46 -04:00
98578b18aa
Update to dynamodb-lock-client 1.2.0
2023-06-21 15:08:40 -04:00
6d81f69785
Start and stop all lifecycle-managed objects in `CrawlAccountsCommand`
2023-06-17 10:17:46 -05:00
7dce183170
Add worker thread pool to PushFeedbackProcessor
2023-06-16 11:36:28 -05:00
f1962a03ef
Parameterize worker thread count in CrawlAccountsCommand
2023-06-16 11:36:28 -05:00
befd336372
Remove static Remote Config auth tokens
2023-06-15 12:11:20 -05:00
8501e61eb1
Set maxThreads = minThreads on command thread pools
2023-06-15 12:11:10 -05:00
ae489e5a52
Log account ages when identity keys change
2023-06-15 13:10:35 -04:00
13afdbda97
Report system resource metrics from background tasks
2023-06-14 16:48:23 -05:00
9cfd88a23f
Move turn secret to static configuration
2023-06-14 10:47:17 -04:00
13456bad3a
Update date math with JSR310 types
2023-06-14 10:47:17 -04:00
45be85c5ef
Update formatting and resolve warnings/suggestions
2023-06-14 10:47:17 -04:00
861dc0d021
reject message sends that have the same device more than once
2023-06-13 09:49:50 -07:00
128d709c99
Additional counters and timers for WebSocket connections
2023-06-13 11:46:15 -05:00
e8f01be8ef
Inject version bytes if missing from existing EC pre-keys
2023-06-09 11:41:51 -04:00
7f1ee015d1
Treat blank strings as null pre-keys
2023-06-09 10:39:16 -04:00
17aa5d8e74
Use strongly-typed pre-keys
2023-06-09 10:08:49 -04:00
b27334b0ff
Treat blank strings as null identity keys
2023-06-09 10:08:18 -04:00
7fc6b1e802
Count invalid pre-keys
2023-06-09 09:25:31 -04:00
8ec6a24a2d
Fix a metric name/tag set
2023-06-08 12:34:27 -04:00
234707169e
Represent identity keys as `IdentityKey` instances
2023-06-08 11:36:58 -04:00
1c8443210a
Check for missing version bytes in invalid identity keys
2023-06-08 09:56:21 -04:00
aaf43a592f
Replace reserved "notification" key with "newMessageAlert"
2023-06-08 09:53:31 -04:00
2b08742c0a
Create separate key stores for different kinds of pre-keys
2023-06-06 17:08:26 -04:00
cac04146de
Identify specific cases with invalid identity keys
2023-06-06 17:08:01 -04:00
2b266c7beb
Validate registration ids for new accounts
2023-06-06 11:08:54 -04:00
099932ae68
ApnPushNotifcationScheduler: always run worker thread at least once
2023-06-06 11:04:44 -04:00
8579babde6
Count instances where an account's identity key could not be interpreted as an `IdentityKey`
2023-06-06 11:01:25 -04:00
9c93d379a8
Fix a sneaky merge conflict
2023-06-05 12:38:35 -04:00
085c7a67c8
Refactor account locks/deleted account manager
2023-06-05 12:30:44 -04:00
e6917d8427
minor cleanup, docs, and integration tests for username API
2023-06-02 10:35:07 -07:00
47cc7fd615
username links API
2023-06-02 10:26:14 -07:00
ecd207f0a1
Check structural validity of prekeys at upload time
2023-05-31 14:29:39 -07:00
d1e38737ce
Support ID token at `PUT /v1/config` and `DELETE /v1/config`
2023-05-30 10:28:28 -05:00
f17de58a71
Change `ScheduledApnPushNotificationSenderServiceCommand` to extend `ServerCommand`
2023-05-30 10:14:33 -05:00
dd552e8e8f
Change `MessagePersisterServiceCommand` to extend `ServerCommand`
2023-05-30 10:14:33 -05:00
18480e9d18
Move metrics registry `environment.manage()` to utility
2023-05-30 10:14:33 -05:00
7ffccd9c3a
Initialize metrics in `ScheduledApnPushNotificationSenderServiceCommand`
2023-05-26 16:41:17 -05:00
0edd99e9cf
Initialize metrics in `MessagePersisterServiceCommand`
2023-05-26 16:41:17 -05:00
defdc14d5e
Initialize metrics in `CrawlAccountsCommand`
2023-05-26 16:41:17 -05:00
5dcf8edd38
Factor metrics registry intialization to a utility
2023-05-26 16:41:17 -05:00
91805caa9a
Finalize rate limit unit inversion
2023-05-26 14:17:30 -04:00
48d39dccbd
Fix rate limit division errors
2023-05-26 13:13:02 -04:00
e7bc8bd6b9
Consistently use AWS credentials providers from `WhisperServerService`
2023-05-26 12:45:38 -04:00
f513dc0398
Invert rate limit units in default configurations
2023-05-26 11:37:06 -04:00
184969336e
Allow `RateLimiterConfig` to accept either a leak rate per minute or a permit regeneration duration
2023-05-26 11:37:06 -04:00
1534f1aa6a
Add web identity token AWS SDK credentials provider
2023-05-26 11:07:41 -04:00
cd8f74e60b
Add support for environment-dependent secondary OAuth2 credentials JSON
2023-05-26 11:07:30 -04:00
d832eaa759
Represent identity keys as byte arrays
2023-05-26 10:12:22 -04:00
796863341d
Revert "Count identity keys that are present, but can't be parsed as base64"
...
This reverts commit 024dd02628a7d989424273501528b52fe18c3ee9.
2023-05-26 10:12:22 -04:00
217b68a1e0
Represent pre-key public keys and signatures as byte arrays in DAOs
2023-05-26 09:58:38 -04:00
4a8ad3103c
Actually write pre-keys as byte arrays
2023-05-26 08:23:54 -04:00
a5f853c67a
Change inactive account age threshold from 365 to 180 days
2023-05-26 08:23:19 -04:00
70b54e227e
Count the prevalence of keys stored as strings or as bytes
2023-05-25 10:04:38 -05:00
1ab6bff54e
add `@Produces` annotations to a few methods in DeviceController
2023-05-25 09:57:06 -05:00
c2317e8493
Start the dynamic configuration manager in dependent commands
2023-05-25 09:52:01 -05:00
b034a088b1
Add support for "atomic" device linking/activation
2023-05-19 16:13:37 -04:00
ae7cb8036e
Factor `DeviceActivationRequest` out into its own record
2023-05-19 16:13:37 -04:00
8ce2b04fe4
Discard test device codes
2023-05-19 15:57:14 -04:00
a3c37aed47
Remove obsolete field from `SecureValueRecovery2Configuration`
2023-05-19 15:57:01 -04:00
fa8f19fd43
Group atomic account creation operations
2023-05-19 15:56:45 -04:00
c9a9409b9a
Count identity keys that are present, but can't be parsed as base64
2023-05-19 15:56:27 -04:00
d3e0ba6d44
Prepare to read pre-keys stored as byte arrays
2023-05-19 15:56:13 -04:00
300ac16cf1
Handle "transport not allowed" responses from the registration service
2023-05-19 15:55:53 -04:00
3e53884979
Add `MessagePersisterServiceCommand`
2023-05-18 15:37:54 -05:00
6043c1a4e8
Add `ScheduledApnPushNotificationSenderServiceCommand`
2023-05-18 11:44:01 -05:00
66a619a378
Allow for atomic account creation and activation
2023-05-18 09:51:13 -04:00
Katherine Yen