Commit Graph

2350 Commits

Author SHA1 Message Date
Ravi Khadiwala 9e510a678c disable response buffering on the websocket
Jersey buffers responses (by default up to 8192 bytes) just so it can
add a content length to responses. We already buffer our responses to
serialize them as protos, so we can compute the content length
ourselves. Setting the buffer to zero disables buffering.
2024-03-08 13:46:00 -06:00
Chris Eager 2dc0ea2b89 Address potential NullPointerException when calling Collection#contains 2024-03-08 13:43:31 -06:00
Alex Konradi 7d364ca7ce
Count group auth reqs without pniAsServiceId=true 2024-03-06 15:41:48 -05:00
Chris Eager 9f6a6d7f5b Include HTTP/2 stream idle timeouts in IOExceptionMapper 2024-03-06 11:11:39 -06:00
Ravi Khadiwala 3cc740cda3 Temporarily hold a username after an account releases it 2024-03-06 11:09:25 -06:00
Ravi Khadiwala 47b24b5dff Simplify username operations in `Accounts`
- Group username table constants together
- Rethrow JsonProcessingException earlier
- Use UpdateAccountSpec.forAccount in username operations
- Inline confirm/clear transaction helpers
2024-03-06 11:09:25 -06:00
adel-signal 8f100a792e
calling: update TurnCallRouter to reduce returned options 2024-03-05 11:26:19 -08:00
Ravi Khadiwala 4e5dd914dd Add varargs variant to HmacUtil 2024-02-28 15:18:39 -06:00
ravi-signal 2adf1e5017
Avoid modification of Account from `@ReadOnly` endpoint 2024-02-28 15:18:12 -06:00
Jon Chambers d1d03f45c5 Resolve warnings throughout `AccountControllerTest` 2024-02-26 16:11:03 -05:00
Ravi Khadiwala 436bd891bd Don't register SpamFilter as a request filter 2024-02-26 09:33:36 -06:00
Chris Eager a7c28fe5ed Rename "name" tags to be more distinguishing 2024-02-26 09:32:50 -06:00
Chris Eager 60814d1ff0 Configure metrics registries earlier in background command setup 2024-02-26 09:32:01 -06:00
Jon Chambers d018efe2a5
Require non-null proofs in "confirm username hash" requests 2024-02-26 10:30:52 -05:00
Jon Chambers 6fd0cba06a Temporarily restore the `giftBadges` capability for backward compatibility 2024-02-26 10:30:29 -05:00
Katherine 12c6af23ee
Map TransactionConflict to ContestedOptimisticLockException in username flows 2024-02-23 14:06:03 -08:00
Ravi Khadiwala 69330f47fd Explicitly call spam-filter for verification session updates
Pass in the same information to the spam-filter, but just use explicit
method calls rather than jersey request filters.
2024-02-23 16:04:24 -06:00
Ravi Khadiwala 4f40c128bf Explicitly call spam-filter for challenges
Pass in the same information to the spam-filter, but just use explicit
method calls rather than jersey request filters.
2024-02-23 15:58:52 -06:00
Jon Chambers 30b5ad1515 Fix an incorrectly-named "check keys" method 2024-02-23 13:17:10 -05:00
adel-signal 665a26d164
update call routing to return ipv6 ips in RFC3986 format 2024-02-23 11:57:58 -05:00
Jon Chambers a5774bf6ff
Introduce a (dormant) Noise/WebSocket for future client/server communication 2024-02-23 11:42:42 -05:00
Jon Chambers d2716fe5cf Add an endpoint for checking that clients and the server have a common view of the client's repeated-use keys 2024-02-23 11:41:58 -05:00
Jon Chambers 279f877bf2 Validate pre-key signatures via the legacy "set signed pre-key" endpoint 2024-02-23 11:33:41 -05:00
Ravi Khadiwala df69d9f195 Annotate authenticated endpoints with `@ReadOnly` or `@Mutable` 2024-02-22 12:05:32 -06:00
Ravi Khadiwala 26ffa19f36 Lifecycle management for Account objects reused accross websocket requests 2024-02-22 12:05:32 -06:00
Jon Chambers 29ef3f0b41 Add tests and metrics for parsing invalid keys 2024-02-21 15:32:25 -05:00
Jon Chambers 106d5e54c7 Extract a common base class for public key serializers/deserializers 2024-02-21 15:32:25 -05:00
Jon Chambers 6ac2460eb0 Drop the "key without version byte" counter 2024-02-21 15:32:25 -05:00
Chris Eager c8d649e8c2 Update MetricsHttpChannelListener to share MetricsRequestEventListener namespace 2024-02-21 13:32:47 -06:00
Ravi Khadiwala 1fdf82dd6c Remove unused ChangePhoneNumberRequest object
Actual request object is ChangeNumberRequest
2024-02-21 13:32:27 -06:00
Jon Chambers 4aa4246695 Clear account records from the account cache after username operations 2024-02-20 12:11:29 -05:00
Chris Eager 1bebceb29c Actually fix flaky test in MetricsHttpChannelListenerIntegrationTest 2024-02-20 10:52:04 -06:00
Jon Chambers a2139ee236 Revert "Add diagnostic dimensions to the "get keys" counter"
This reverts commit cd64390141.
2024-02-18 20:14:05 -05:00
Jon Chambers 8c55f39cdf Revert "Use a phased enrollment strategy for the `pnp` compatibility flag"
This reverts commit 3e12a8780d.
2024-02-18 20:08:39 -05:00
Chris Eager 0329184c94 Fix flaky test in MetricsHttpChannelListenerIntegrationTest 2024-02-18 20:08:29 -05:00
Jon Chambers cd64390141 Add diagnostic dimensions to the "get keys" counter 2024-02-18 18:10:58 -05:00
Jon Chambers 3e12a8780d Use a phased enrollment strategy for the `pnp` compatibility flag 2024-02-18 17:59:00 -05:00
Jon Chambers 11e6ff1bbe Introduce a `pnp` capability for backward compatibility 2024-02-18 17:59:00 -05:00
Jon Chambers 36f85fc97e Fix an inverted boolean in a counter 2024-02-16 15:18:18 -05:00
Jon Chambers 9040cfd200 Count "store key" attempts by key type/platform 2024-02-16 13:09:55 -05:00
Jon Chambers 757da3b15a Fully-qualify OpenAPI `Tag`, allowing for imported Micrometer `Tag` 2024-02-16 13:09:55 -05:00
Jon Chambers d162590a32 Record a remote-aggregated distribution of one-time pre-key counts 2024-02-16 13:09:55 -05:00
Jon Chambers f41e1716c6 Combine/expand existing "get keys" counter 2024-02-16 13:09:55 -05:00
Jon Chambers 4dce0f1b9d Add identity type/"stale" dimensions to the "pre-key unavailable" counter 2024-02-16 13:09:55 -05:00
Jonathan Klabunde Tomer fef57dce0d
use explicit `Timer` rather than micrometer annotation for send-message latency distribution 2024-02-15 14:58:43 -08:00
Jon Chambers d884700b61 Significantly reduce default Redis command timeout 2024-02-15 17:57:17 -05:00
Jonathan Klabunde Tomer ff9ad4bd1d
export 1:1 message send latency as a full distribution 2024-02-15 10:33:02 -08:00
Chris Eager 9ce2b7555c Add static servlet paths to MetricsHttpChannelListener 2024-02-14 17:09:30 -06:00
Jon Chambers f90ccd3391 Count attempts to fetch a one-time EC pre-key that result in a "key unavailable" response 2024-02-14 18:04:31 -05:00
Jon Chambers 5ff092e541 Retire the `pni` device capability 2024-02-14 18:04:23 -05:00
Ravi Khadiwala dcdf401f64 Make DELETE /v1/device synchronous 2024-02-14 15:30:10 -05:00
Chris Eager e4fb80b39b Add onResponseFailure handler to MetricsHttpChannelListener 2024-02-14 13:43:23 -06:00
Chris Eager 9745854ab8 Switch MetricsHttpChannelListener to ContainerResponseFilter 2024-02-14 13:43:23 -06:00
Jon Chambers 7124621f66
Use pre-calculated pre-key counts when possible 2024-02-14 14:28:33 -05:00
Jon Chambers 47fd8f5793 Assume that all devices are PNI-capable 2024-02-14 14:27:48 -05:00
Jonathan Klabunde Tomer 40d698f2db Remove last vestiges of stale capabilities.
Not for deployment until 2024-02-06!
2024-02-14 14:23:05 -05:00
adel-signal 74abe98706
initialize call routing data syncs 2024-02-13 17:05:08 -08:00
Chris Eager 86787f3bc8 Start DynamicConfigurationManager directly after construction 2024-02-13 13:08:20 -06:00
Chris Eager 699b0c775a Add dynamic configuration to enable detailed Lettuce metrics 2024-02-13 10:33:43 -06:00
Chris Eager ff59ef8094 Add MetricHttpChannelListener 2024-02-13 10:29:03 -06:00
Jonathan Klabunde Tomer 089af7cc1f
correct metric name in MaxMindDatabaseManager timer 2024-02-09 13:37:39 -08:00
Jonathan Klabunde Tomer 1591a2d9a3 CallRoutingTableManager: fix crash on startup due to typo in metric naming 2024-02-09 12:27:51 -08:00
Jonathan Klabunde Tomer f7984ed642
CallDnsRecordsManager: fix crash on startup due to typo in metric naming 2024-02-09 12:14:50 -08:00
adel-signal d1f68eacd9
Add call routing API endpoint for turn servers 2024-02-09 11:12:23 -08:00
Jon Chambers 4f45f23094 Prevent old versions of the Android app from carrying out "change phone number" requests 2024-02-09 09:58:38 -05:00
Jon Chambers c5dc01ee11 Restore high-cardinality Lettuce metrics for debugging 2024-02-08 19:27:56 -05:00
Ravi Khadiwala 3b44ed6d16 Explicitly call spam-filter for messages
Pass in the same information to the spam-filter, but just use explicit
method calls rather than jersey request filters.
2024-02-07 14:30:32 -06:00
Chris Eager 0965ab8063 Move HeaderUtils#getMostRecentyProxy to RemoteAddressFilter 2024-02-07 14:28:37 -06:00
Fedor Indutny 24a7762873
Add `identityType` tag to `SENT_MESSAGE_COUNTER_NAME` 2024-02-06 17:39:00 -06:00
Chris Eager e441ab60a2 Add metric for IPv4/IPv6 requests count 2024-02-06 17:38:25 -06:00
Chris Eager 2ab14ca59e Refactor remote address/X-Forwarded-For handling 2024-02-05 13:37:06 -06:00
Chris Eager 4475d65780 Make Accounts#changeNumber exception handling more resilient to future changes 2024-02-01 15:14:15 -06:00
Ravi Khadiwala b1d10f5817 Add lifecycle management for AwsCrt for commands 2024-02-01 15:08:45 -06:00
Chris Eager c838df90ef Add HttpServletRequestUtil 2024-01-31 17:53:47 -06:00
Chris Eager fb39af67e5 Allow for more TransactWriteItems when inspecting TransactionCanceledException 2024-01-31 17:46:32 -06:00
Chris Eager 2d4d37f96a Translate TransactionCanceledException to ContestedOptimisticLockException in Accounts#changeNumber 2024-01-31 17:27:16 -06:00
Jon Chambers 26adf20ee8 Make "fetch profile" endpoints `@ManagedAsync` 2024-01-31 14:38:50 -06:00
Jon Chambers 72668ed0a2 Make "send individual message" endpoint `@ManagedAsync` 2024-01-31 14:38:50 -06:00
Ravi Khadiwala 50f1ed7851 Add a virtual-thread backed `@ManagedAsyncExecutor` 2024-01-31 14:38:50 -06:00
ravi-signal cf8f2a3463
remove synchronized locks that may be held while blocking 2024-01-31 14:29:15 -06:00
ravi-signal b483159b3a
reintroduce VirtualThreadPinEventMonitor 2024-01-31 14:28:12 -06:00
ravi-signal 480abebf7e
Offload account lock updates to accountLockExecutor 2024-01-31 14:27:16 -06:00
Ravi Khadiwala b924dea045 Remove VirtualThreadPinEventMonitor 2024-01-30 13:49:39 -06:00
Ravi Khadiwala 2c1e7e5ed6 Disable flaky VirtualThreadPinEventMonitorTest 2024-01-30 13:29:58 -06:00
Ravi Khadiwala 4dfd74906c Add timeouts to VirtualThreadPinEventMonitorTest 2024-01-30 13:10:43 -06:00
Chris Eager c80225a18c Remove `NoneHealthResponder` 2024-01-30 12:51:43 -06:00
ravi-signal 0e6242373e
Add a monitor for virtual thread pin events 2024-01-30 12:48:07 -06:00
ravi-signal 4305db5579
Update to aws sdk 2.23.8 and use AwsCrtHttpClient 2024-01-30 12:46:27 -06:00
ravi-signal 36e7772f74
Put some validation on to profile version strings
Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>
2024-01-26 15:24:38 -06:00
Chris Eager ca05df5172 Revert "Add support for secondary credentials for registration service"
This reverts commit 4b8fc2950f.
2024-01-26 11:42:35 -06:00
Chris Eager 422e8e6f3e Remove CommandStopListener 2024-01-26 11:31:04 -06:00
Jon Chambers 852b285d84 Configure and instantiate a `ClientPublicKeys` data store/manager 2024-01-26 10:29:59 -05:00
Jon Chambers 6c13193623 Add a data store for client public keys for transport-level authentication/encryption 2024-01-26 10:29:59 -05:00
Ravi Khadiwala 6bda9d8604 Set destination account on the request context 2024-01-25 13:02:28 -06:00
Ravi Khadiwala 1428ca73de Make identity token fetcher more async friendly
After the identity token expires a subsequent call would do a blocking
operation to retrieve the new token. Since we're making use of an async
gRPC client, this tends to block a thread we don't want to be blocking
on.

Instead, switch to periodically refreshing the token on a dedicated
thread.
2024-01-25 13:01:21 -06:00
Katherine 498ace0488
Remove iOS special case handling logic for SEPA/iDEAL 2024-01-25 10:52:17 -08:00
Chris Eager 4b8fc2950f Add support for secondary credentials for registration service 2024-01-24 13:52:23 -06:00
Jon Chambers 595cc55578 Retire the `returnPqKey` flag when fetching pre-keys 2024-01-23 16:01:40 -05:00
Ravi Khadiwala 91b0c368b4 Remove unused table configuration parameter 2024-01-23 14:48:52 -06:00
Jonathan Klabunde Tomer 21d0ffc990
Allow "reserving" one's own committed username hash 2024-01-23 12:44:55 -08:00
Chris Eager 55b9d84956 Add `host` to `DogstatsdConfiguration` 2024-01-22 15:59:52 -06:00
Chris Eager ffdb0db6c6 Migrate from `host` tag to `dd.internal.entity_id` 2024-01-22 15:59:52 -06:00
Chris Eager a5ed07a666 Support environment variable substitution in configuration 2024-01-22 11:25:18 -06:00
Chris Eager da02c90bad Remove AWS instance profile credentials provider 2024-01-22 11:24:03 -06:00
Ravi Khadiwala 3820a231ec Strictly enforce ACI service identifier strings have no prefix 2024-01-22 11:23:33 -06:00
Dimitris Apostolou 408b065b9e Fix typos 2024-01-20 12:40:08 -05:00
hduelme 238ab84749 use StandardCharsets UTF_8 instead of String 2024-01-20 12:39:43 -05:00
Ravi Khadiwala f5080f9bd6 Support configurable verification code sender overrides 2024-01-19 13:58:17 -06:00
Ravi Khadiwala db4aa99ce0 Make the ServiceIdentifier interface sealed 2024-01-19 13:57:20 -06:00
ravi-signal 70134507f8
Add metric for ServiceId string parsing 2024-01-19 13:56:47 -06:00
Ravi Khadiwala 360a4793ae Remove global lock in RateLimitChallengeManager 2024-01-19 13:56:09 -06:00
Jon Chambers 47bfb25f2c Retire a now-unused transacational update method for accounts 2024-01-18 16:40:21 -05:00
Jon Chambers b048b0bf65 Remove signed pre-keys from `Device` entities 2024-01-18 12:13:00 -05:00
Jonathan Klabunde Tomer 394f9929ad
limit size of multi-recipient messages 2024-01-11 16:31:37 -08:00
ravi-signal bf39be3320
Add Content-Type header for copy uploads 2024-01-11 14:59:35 -06:00
Chris Eager 4a2cbb9ec7 Stop timers on Flux termination 2024-01-11 14:57:31 -06:00
Ravi Khadiwala cc6cf8194f Add media deletion endpoint 2024-01-11 10:19:06 -06:00
Ravi Khadiwala e934ead85c Fix incorrect comparison in archive usage metric 2024-01-11 10:19:06 -06:00
Ravi Khadiwala 323bfd9a6e Remove some secondary keys from account cache
Remove e164, usernameHash, and usernameLink secondary mappings from the
accounts redis cache.
2024-01-11 10:10:32 -06:00
Jon Chambers bf05e47e26 Get accounts asynchronously when checking SVR credentials via gRPC 2024-01-11 11:09:49 -05:00
Jon Chambers d18f576239 Revert "Revert "Treat the stand-alone signed pre-keys table as the source of truth for signed pre-keys""
This reverts commit 3f9edfe597.
2024-01-10 11:22:10 -05:00
Jon Chambers 7d483c711a Revert "Revert "Retire "migrate signed pre-keys" configuration""
This reverts commit a024949311.
2024-01-10 11:22:10 -05:00
Jon Chambers 61256d49cd Revert "Revert "Retire the "migrate signed pre-keys" command""
This reverts commit f738bc97e7.
2024-01-10 11:22:10 -05:00
Jonathan Klabunde Tomer 184cdc0331
Remove two-stage check of username availability in reserve/confirm 2024-01-09 14:01:42 -08:00
Ravi Khadiwala ed972a0037 Fix archive listing directory prefix order 2024-01-09 16:01:11 -06:00
Ravi Khadiwala a62a6c1cb6 Change type of Cdn3StorageManagerConfiguration.clientSecret to SecretString 2024-01-08 16:06:54 -06:00
Ravi Khadiwala f66566aa17 Fix default configuratiton in MonitoredS3ObjectConfiguration 2024-01-08 14:52:14 -06:00
ravi-signal b6ecfc7131
Add archive listing 2024-01-08 13:54:57 -06:00
adel-signal 2b688b1a60
Refactor spam filter's S3MonitoredObject to server parent module.
Allows other parts of server to use S3MonitoredObjects.
2024-01-08 10:34:12 -08:00
Jon Chambers 3c64d9292f Revert "Expand the default max packet size for Dogstatsd"
This reverts commit 0f52d2e464.
2024-01-08 12:23:16 -05:00
Jon Chambers 0f52d2e464 Expand the default max packet size for Dogstatsd 2024-01-08 11:43:50 -05:00
Jonathan Klabunde Tomer 1e5fadc440
Allow reserving a hash previously reserved (but not committed) by the same user 2024-01-05 15:38:45 -08:00
Chris Eager f495ff483a
Update RemoveExpiredLinkedDevicesCommand to retry failures 2024-01-05 15:38:34 -08:00
Jonathan Klabunde Tomer d1a80cc880 fix build-breaking typo 2024-01-04 11:46:07 -08:00
Jonathan Klabunde Tomer e1ad25cee0
Group Send Credential support in chat 2024-01-04 11:38:57 -08:00
Chris Eager 195f23c347
Add `/v1/accounts` and `/v2/keys` to the WebSocket 2024-01-02 15:51:57 -06:00
Chris Eager ad6b99be6a Dispatch client presence operations after device deletion to a dedicated executor 2023-12-22 10:51:17 -06:00
Chris Eager b9dd9fc47d Reduce fan-out by processing a single stream of expired linked devices 2023-12-22 10:51:06 -06:00
Chris Eager 637792c6d4 Move `RemoveExpiredLinkedDevicesCommand` error handling for more accurate metrics 2023-12-21 16:06:41 -06:00
Jon Chambers 4d1bca2d97 Dispatch client presence operations after account deletion to a dedicated executor 2023-12-21 13:40:49 -05:00
Chris Eager f33a2eba50
Add buffer + shuffle and configurable concurrency to `RemoveExpiredLinkedDevicesCommand` 2023-12-21 11:14:02 -06:00
Jon Chambers 5d6bea5ec9 Clear Redis cache entries strictly after removing accounts from DynamoDB 2023-12-21 12:08:03 -05:00
Jon Chambers 057d1f07a8 Remove bulk "set repeated-use signed pre-keys" methods because they were only ever used for single devices 2023-12-21 12:07:42 -05:00
Jon Chambers 95bb9a9780 Log signed pre-key retrieval errors and mismatches 2023-12-20 14:55:12 -05:00
Chris Eager 06c391cbf6 Add counter for failed account updates to RemoveExpiredLinkedDevicesCommand 2023-12-20 13:51:28 -06:00
Chris Eager d90dff95b1 Move MeterRegistry#close calls to after the lifecycle has fully stopped 2023-12-20 13:51:16 -06:00
Ravi Khadiwala ca47a7b663 handle new RegistrationService proto error 2023-12-19 18:19:26 -06:00
Jonathan Klabunde Tomer 9d3d4a3698
Add phone-number-sharing field to versioned profile
Co-authored-by: Katherine <katherine@signal.org>
2023-12-19 11:20:04 -08:00
Chris Eager 3b509bf820 Add command to remove expired linked devices 2023-12-19 13:11:26 -06:00
Jon Chambers 5b7f91827a
Remove signed pre-keys transactionally when removing devices 2023-12-19 14:11:05 -05:00