Multiple insertion management

[nicolasbraun]

Hello @demchenkoalex

I encountered an issue that might be worth discussing.

Basically my chat controller manages a List of Message.
When loading a previous page / a catchup page when coming from background I opted to insert them using several ChatOperation.insert because looking at _onDiffUpdate

flutter_chat_ui/packages/flutter_chat_ui/lib/src/chat_animated_list/chat_animated_list_reversed.dart

Line 618 in 9b3d6e4

void _onDiffUpdate(diffutil.DataDiffUpdate<Message> update) {

it basically does that but has to loop on all the messages already in the list so if it gets big , and since I know where to insert, I feel it will be way more performant.

My problem is since we listen to a Stream of Operations it's async so when you do

flutter_chat_ui/packages/flutter_chat_ui/lib/src/chat_animated_list/chat_animated_list_reversed.dart

Line 167 in 9b3d6e4

_oldList = List.from(_chatController.messages);

the chatController's messages might already contain all the messages when processing the first (or any) insert.

This actually breaks my PR #754 because _oldList.length is bigger than the _listKey.currentState, causing a crash.

I was thinking we should only rely on the local state (made a test here but this also has drawbacks: since Operations are executed in parralel we might end up with ops overwritting _oldList. It think it's only fixable with a queue of operations, but once again i'm not a Dart pro.

[demchenkoalex]

Should we just introduce insertMany with index? Also, DiffUtil performs well with around 500–1000 items in the list. I was hoping people wouldn’t go beyond that and would instead delete the oldest messages, then fetch them again if the user actually scrolls there.

[nicolasbraun]

@demchenkoalex That would be a nice addition ! Usually the client knows where to insert (at start or beginning of the list) so this would ensure good performance for very big chats, especially now that v2 easily handles Sembast or any other local storage (so all previous messages will be there)

But i'm not sure this solves the core issue: The stream of Operations being async it makes it risky to rely on chatController's messages. This fix in #754 will never be safe because at some point we might try to insertItem in the scrollview at an idem bigger than it's _itemCount. I see those options (but once again I'm quite new to Flutter)

1. Quick and dirty

• just try catch the insert => since anyway the list currently rebuild with chatController.messages in the initialItemsCount and builder we don't have any animation anyway

2.Expect the operation to be quick enough
and rely on localState (like I did in my test branch). But If for some reason an op takes long (scenario of a big set op followed by just a new message insertion , which can easily happen when chat loads past and start listening to new messages) we will loose messages. I don't like it

Best? : A queue of ops

Something like

List<ChatOperation> _operationsQueue;
bool _isProcessingOpsQueue = false;


  void _processQueue()  {
    // Safety to no process twice, should not really happen
    if (_isProcessingOpsQueue) return;
    _isProcessingOpsQueue = true;
      while (_operationsQueue.isNotEmpty) {
        final ops = List.of(_operationsQueue);
        _operationsQueue.clear();
       // process ops synchronously
      }
   _isProcessingOpsQueue = false;  
  }

and on new stream emit we insert in the queue + call processQueue

[nicolasbraun]

@demchenkoalex i'm cooking something, i'll make a PR tomorrow that we can discuss

[demchenkoalex]

nice 🙌 this week is super busy at my job, can't look myself into this yet..

[demchenkoalex]

should be solved by #756