DataStore subscriptionProcessor failed

[mvishnureddy007]

Before opening, please confirm:

• I have searched for duplicate or closed issues and discussions.

Language and Async Model

Java

Amplify Categories

Authentication, GraphQL API, DataStore

Gradle script dependencies

implementation 'com.amplifyframework:core:2.13.0'
implementation 'com.amplifyframework:aws-api:2.13.0'
implementation 'com.amplifyframework:aws-datastore:2.13.0'
implementation 'com.amplifyframework:aws-auth-cognito:2.13.0'
implementation 'com.amplifyframework:rxbindings:2.13.0'

Environment information

implementation 'com.amplifyframework:core:2.13.0' implementation 'com.amplifyframework:aws-api:2.13.0' implementation 'com.amplifyframework:aws-datastore:2.13.0' implementation 'com.amplifyframework:aws-auth-cognito:2.13.0' implementation 'com.amplifyframework:rxbindings:2.13.0'

Please include any relevant guides or documentation you're referencing

No response

Describe the bug

Hi,
We are getting this error after user has been logged in and out for multiple times.

DataStoreException{message=DataStore subscriptionProcessor failed to start., cause=GraphQLResponseException{message=Subscription error for MessageRecipient: [GraphQLResponse.Error{message='subscription exceeds maximum value limit 20 for operator containsAny.', locations='null', path='null', extensions='{errorCode=400}'}], errors=[GraphQLResponse.Error{message='subscription exceeds maximum value limit 20 for operator containsAny.', locations='null', path='null', extensions='{errorCode=400}'}], recoverySuggestion=See attached list of GraphQLResponse.Error objects.}, recoverySuggestion=Check your internet.}

GraphQLResponseException{message=Subscription error for MessageRecipient: [GraphQLResponse.Error{message='subscription exceeds maximum value limit 20 for operator containsAny.', locations='null', path='null', extensions='{errorCode=400}'}], errors=[GraphQLResponse.Error{message='subscription exceeds maximum value limit 20 for operator containsAny.', locations='null', path='null', extensions='{errorCode=400}'}], recoverySuggestion=See attached list of GraphQLResponse.Error objects.}

Reproduction steps (if applicable)

No response

Code Snippet

// Put your code below this line.

Log output

// Put your logs below this line

amplifyconfiguration.json

No response

GraphQL Schema

// Put your schema below this line

Additional information and screenshots

No response

[tylerjroach]

@mvishnureddy007 You appear to be running into a limit on the AppSync side. Do you have >20 containsAny predicates?

[mvishnureddy007]

@tylerjroach We don't have containsAny predicate in our code. I'm providing our observer Code here

QueryPredicate predicate = MessageRecipient.USER_ID.eq(ApplicationData.shared.userProfileData.getId());
QuerySortBy sortBy = new QuerySortBy("MessageRecipient", "sentAt", QuerySortOrder.ASCENDING);
List sortByList = new ArrayList();
sortByList.add(sortBy);
ObserveQueryOptions options = new ObserveQueryOptions(predicate, sortByList);
Amplify.DataStore.observeQuery(MessageRecipient.class,
options,
observationStarted,
onQuerySnapshot,
onObservationError,
onObservationComplete);

Our scenario is happening because of Same user logged into multiple devices.

I'm providing schema
type MessageRecipient @model @auth(rules: [
{allow: groups, groupsField: "permittedTo"}
]) {
id: ID!
messageId: ID! @index(name:"byMessage")
groupId: String! @index(name: "byGroup")
userId: Int! @index(name:"byUser")
status: MessageStatus!
sentAt: AWSDateTime!
readAt: AWSDateTime
permittedTo: [String!]
}

[tylerjroach]

Even though the Android client only uses client-side filtering, the service still uses Enhanced Subscription Filtering on the service side to enforce dynamic auth rules.

This rule

@auth(rules: [
  {allow: groups, groupsField: "permittedTo"}
]) 

Gets applied service-side in the subscription auth resolver with the containsAny operator to match the groups the user is in to the groups define in the group field permittedTo. The limit is listed here.

Most likely the root cause is: either some of the records have too many groups listed in the permittedTo field or the other way around, the user is a member of too many groups.

I'm not exactly sure how multiple sign-in/sign-outs, or multiple devices would cause the failure. As far as I understand, the limit is per subscriber.

If the explanation above doesn't help resolve the issue, please reply back with information around how Amplify is configured. How DataStore is started/stopped, and any other information you believe may be relevant, such as specific scenarios around the mentioned multiple device/sign-ins.

[mvishnureddy007]

If the user can be part of more than 20 groups, which is common situation in our workflow.
Can you help us, how to handle this?

[iartemiev]

There's no way to accommodate this when using AppSync subscriptions due to the way subscription filtering is implemented in the service. This is a hard limit in the service and it's not possible to increase it on a case-by-case basis.

You may be able to disable subscriptions for models that use dynamic group auth ({allow: groups, groupsField: "permittedTo"}) and then rely on DataStore sync queries to retrieve new/updated records, similar to a polling approach, but it's not possible to get this working with real-time subscription messages without reducing the number of groups to <= 20.

Note that the same limit applies for a list of owners or any other dynamic, list-based row-level authorization.

[mvishnureddy007]

We have removed this

({allow: groups, groupsField: "permittedTo"})

Subscription is working fine.

But we got another issue. ie., After every fresh login the Data store query sync is taking long time to get data from cloud.
Step we are following

1. Once logged into mobile we are checking auth session.
2. After that we are doing this for start data store query
   Amplify.DataStore.start(
   () -> Log.i("MyAmplifyApp", "DataStore started"),
   error -> Log.e("MyAmplifyApp", "Error starting DataStore", error)
   );
3. Once i go to chat at first time data store query retring empty or 1 record only.