Fairhaven2022 Speeding up the Web Questionnaire

Discussion: Speeding up Matrix Customization

Presenters: Emily M. Bender & Tom Liu

Moderator: Rosetta Pendleton; I paraphrased a bunch.

Slides: Speeding up the web Questionnaire.pdf

About: The Matrix Customization system (Web Questionnaire) can be very slow especially when loading a large choices file. There are also some issues about saving progress in between navigating between subsections of the questionnaire. This is a discussion to mitigate these issues.

Minutes

Overview of Suggestions

• Tom: Suggests adding a loading animation for when the page is loading (context: it takes up to 36 seconds to load a 635 lexicon choices file).
• Tom: Suggests changing the way the page is initialized (example not loading all fields at once).
• Tom: Mentions if we are interested in searching through the web page.
• Tom: Suggests a dialogue box to save & stay before navigating away to another page in the questionnaire.
• Tom: Suggests If we should automatically save the questionnaire choices when the user navigates to different pages.
• Tom: Suggests displaying when the choices file was last saved (although even this may be a blurry indicator of what is saved because users can make multiple changes in seconds).
• Tom: Suggests Improving

Discussion on Rendering & Redesigning

• Luis: Should we keep the file to be loaded on the user side or should we transform the matrix to a grammar development space where each user has their own version of the choices file saved already where they can edit it.
• Emily: That would involve account management which could be a lot of work.
• Tom: The choices file itself is very small (~100kb) the slowness is from the javascript loading the page. It has algorithms to load the front end features (drop boxes, textboxes, etc.)
• Luis: Every lexical entry has to be saved. If we can have asynchronous fetching of lexical entry using a database on UW side to grab the lexicon this may help.
• Emily: The slowness isn't from the fetching of the lexical entries but the slowness of rendering.
• Guy: We could consider only rendering a part of the page and load more as you scroll down.
• Glenn: We can default to collapse all drop boxes when the page loads.
• Olga: We can also collapse all the fields because the default is they are all open.
• Dan: In order for searching to work successfully we need the whole page to be populated (It uses the browser find feature)
• Guy: We may be able to implement our own search (such as a predicate name) and we can search through the lexicon in the choices file which is typically less than 1000 words.
• Olga: TJ is bringing up the fact that we are using a relatively old framework for the ui (cgi) maybe we can use more modern front end frameworks.
• Emily: Do people have wisdom about react, angular, etc.?
• Michael: We can use react but it is a learning curve.
• Eric: It goes in and out of style pretty easily.
• Luis: Everything is a call back in react
• Eric: Is it really common to load in the grammar file and search for things? We could have a UI for loading display on the page and see the page fill in and you can start interacting with it as it fills in.
• Luis: You can use cgi to not render the full page. But you essentially need more callback functions. The problem is the HTML is there right now and it is slow to render on the browser. You could have it so that when you click on some button a cgi program could load a side panel that users could then interact with.
• Eric: I would be surprised if React will be around for 5 more years as javascript new frameworks get developed constantly. But despite this framework being old it still stands the test of time.
• Michael: It would be nice to parse the choices file in a javascript presentation and the presentation is a filter of whats there and render the objects that are needed for the page on the customization., It can clear the objects and generate the ones that are needed. We can move all the python to javascript as well so it is not a mix of javascript and python.
• Emily: Should we be doing DART?
• Luis: NTU moved to using flask because there are things that are easier to do there than cgi and if we are losing the ability to change things easily in cgi it may be helpful to change to flask. But the people maintaining the system may have a hard time with the learning curve again.
• Michael: There is a template called ninja used with ajax that can help render pages.
• Luis: So the simple move will be to have javascript and flask.
• TJ: (from chat) React is developed by Meta and has a large open source community and Angular is developed by Google and has a large open source community. React was released 9 years ago, Angular was released 5 years ago, I don't think they'll be gone in 5 years.
• Tom: I know react and jquery so I can do that but one thing I am concerned with is the validation. IF there are any errors made in the predication when i hit save & stay it will send that choices to the server, and render the validation messages. The choices can not be purely modified in the front end it also needs to be communicated with the backend.
• Liz: I found the outdated-ness of the website turns students away from wanted to use the platform or join the class so I think even if we invest in react and making the appearance nicer and have to do it again 5 years later it may be worth doing.

Discussion on Save & Stay