Publish Graphs as a Research Note

[IshaGupta18]

Is your feature request related to a problem? Please describe.
Once we complete the integration with plots2, one of the features for Public Lab users could be to publish the created graphs as a Research Note.

Propose ideas to implement this feature
This is how I thought of doing it while I was writing my proposal.

However, I am not sure how to go about it exactly.

Do we store the graphs as images and render them in notes or how would we implement the partial thing exactly.

Additional context

I would love it if anyone could suggest a flow of things here. On how should we go about it.

@jywarren @namangupta01 @rexagod @gauravano @sagarpreet-chadha @IgorWilbert

[jywarren]

Hi! Notice how when you drag a CSV into https://stable.publiclab.org/post, you get this power tag:

That's the "rich text" view. In Markdown, it's simply: [graph:/i/21643.csv]

When you hit Publish, you see this:

https://stable.publiclab.org/notes/warren/07-03-2019/test-csv

That's because this power tag, in format [graph:/i/21643.csv], gets converted into the following HTML/JS code:

<canvas class="inline-graph" id="graph-grid-101"></canvas>
<p class="graph-utils"><small><a href="/i/21643.csv"><i class="fa fa-download"></i> CSV</a></small></p>
<style>
.graph-utils {
text-align: right;
}
.graph-utils a,
.graph-utils i {
color: #aaa;
}
</style>
<script>
(function(){
graphUrl("/i/21643.csv", "graph-grid-101");
})()
</script>
</p>

The substitution is done here: https://github.com/publiclab/plots2/blob/50f97ea138acaeb4db641f50be1835886945d2ef/app/models/concerns/node_shared.rb#L38-L54

And the output is driven from this template (as referenced in the substitution method above, which scans through the note body):

https://github.com/publiclab/plots2/blob/50f97ea138acaeb4db641f50be1835886945d2ef/app/views/grids/_graph.html.erb#L1-L16

See how this fits together?

[jywarren]

Similarly, you could create a new graph powertag, like [simple-graph:/i/example.csv] which would get inserted in the same way. Don't worry about the drag-and-drop part yet, I can show you later. (see here for more)

For now, just focus on how the powertag [graph:/i/example.csv] gets converted into a displayed graph. If you make your own template at /app/views/grids/_simple_graph.html.erb you can start setting this up!

We actually have a test demonstrating the existing graph integration here:

https://github.com/publiclab/plots2/blob/50f97ea138acaeb4db641f50be1835886945d2ef/test/unit/node_shared_test.rb#L170-L174

You could make a similar test to show your own powertag!

[IshaGupta18]

Wow this is some pretty great stuff here. I can see how this ties up together, but I think I might want to do it a little differently. Since the graphs are already made, we could create a Research Note similar to this one but with the already plotted graphs, instead of drag and drop (since it already exists). Let me study this a little more deeply and I think we'll be good to go with a similar thing then. I'll post more doubts once I begin with this! Thanks a ton!

[IgorWilbert]

@IshaGupta18 not sure if I understand how you want to do this. If there is no drag and drop to create a Research Note with a given graph, do you plan to attach the graph to the note with an "upload button", for example? If yes, why don't we do both? If I am not mistaken, in Image Sequencer it is possible to upload an image either selecting the file or using drag and drop. Maybe we could do the same but with graphs and "loading a preview" of them too. Also, if you need some help with UI or coding problems just ping ;)

[jywarren]

I think we could start by implementing the [simple-graph:____.csv] filtering in wikis/notes, without also doing drag/drop at the same time, as a first step. We could for example, as a first step, make it so you have to just drag/drop a CSV as you do now, but then change [graph:____.csv] to [simple-graph:____.csv] and then your code would start working. Then later we could decide how we want to handle the dragging and dropping! How's that sound?

[IshaGupta18]

That sounds pretty good, although a little tough, since we need to run the entire thing for the selection of columns and stuff in the drag drop box itself. What I suggest is, we create a "Publish as research note" button where the graphs are plotted, and from there itself it can pick up the plotted graphs and proceed in a similar manner as above. What do you think?

…

On Mon, Jul 8, 2019, 9:34 PM Jeffrey Warren ***@***.***> wrote: I think we could start by implementing the [simple-graph:____.csv] filtering in wikis/notes, without also doing drag/drop at the same time, as a first step. We could for example, as a first step, make it so you have to just drag/drop a CSV as you do now, but then change [graph:____.csv] to [simple-graph:____.csv] and then your code would start working. Then later we could decide how we want to handle the dragging and dropping! How's that sound? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#65?email_source=notifications&email_token=AJXHQZZQHYSGPQ2GDIG44HTP6NQRTA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNR7WY#issuecomment-509288411>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJXHQZYGNLBLGTSDH7PQ2MTP6NQRTANCNFSM4H5H5UNA> .

[jywarren]

Oh, i see. So the configuration of SDG would need to be added to the short-code perhaps? How would that work? What are the parameters that would need to be stored based on the selection of columns and such?

…

On Mon, Jul 8, 2019 at 12:14 PM Isha Gupta ***@***.***> wrote: That sounds pretty good, although a little tough, since we need to run the entire thing for the selection of columns and stuff in the drag drop box itself. What I suggest is, we create a "Publish as research note" button where the graphs are plotted, and from there itself it can pick up the plotted graphs and proceed in a similar manner as above. What do you think? On Mon, Jul 8, 2019, 9:34 PM Jeffrey Warren ***@***.***> wrote: > I think we could start by implementing the [simple-graph:____.csv] > filtering in wikis/notes, without also doing drag/drop at the same time, as > a first step. We could for example, as a first step, make it so you have to > just drag/drop a CSV as you do now, but then change [graph:____.csv] to > [simple-graph:____.csv] and then your code would start working. Then > later we could decide how we want to handle the dragging and dropping! > How's that sound? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < #65?email_source=notifications&email_token=AJXHQZZQHYSGPQ2GDIG44HTP6NQRTA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNR7WY#issuecomment-509288411 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AJXHQZYGNLBLGTSDH7PQ2MTP6NQRTANCNFSM4H5H5UNA > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#65?email_source=notifications&email_token=AAAF6JZ2DMB4XJ6FHBIXBWDP6NRUXA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNS25Y#issuecomment-509291895>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAF6J2ILZWSFCJVP6XGVTDP6NRUXANCNFSM4H5H5UNA> .

[IshaGupta18]

Yeah, we will create or use the current function for plotting the graph, which already has the required selections, so I believe that should work? On Mon, Jul 8, 2019, 9:50 PM Jeffrey Warren <notifications@github.com> wrote:

…

Oh, i see. So the configuration of SDG would need to be added to the short-code perhaps? How would that work? What are the parameters that would need to be stored based on the selection of columns and such? On Mon, Jul 8, 2019 at 12:14 PM Isha Gupta ***@***.***> wrote: > That sounds pretty good, although a little tough, since we need to run the > entire thing for the selection of columns and stuff in the drag drop box > itself. What I suggest is, we create a "Publish as research note" button > where the graphs are plotted, and from there itself it can pick up the > plotted graphs and proceed in a similar manner as above. What do you think? > > On Mon, Jul 8, 2019, 9:34 PM Jeffrey Warren ***@***.***> > wrote: > > > I think we could start by implementing the [simple-graph:____.csv] > > filtering in wikis/notes, without also doing drag/drop at the same time, > as > > a first step. We could for example, as a first step, make it so you have > to > > just drag/drop a CSV as you do now, but then change [graph:____.csv] to > > [simple-graph:____.csv] and then your code would start working. Then > > later we could decide how we want to handle the dragging and dropping! > > How's that sound? > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > < > #65?email_source=notifications&email_token=AJXHQZZQHYSGPQ2GDIG44HTP6NQRTA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNR7WY#issuecomment-509288411 > >, > > or mute the thread > > < > https://github.com/notifications/unsubscribe-auth/AJXHQZYGNLBLGTSDH7PQ2MTP6NQRTANCNFSM4H5H5UNA > > > > . > > > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < #65?email_source=notifications&email_token=AAAF6JZ2DMB4XJ6FHBIXBWDP6NRUXA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNS25Y#issuecomment-509291895 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AAAF6J2ILZWSFCJVP6XGVTDP6NRUXANCNFSM4H5H5UNA > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#65?email_source=notifications&email_token=AJXHQZ6LJZHOYZEZV7PO4L3P6NSLXA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNTL2Y#issuecomment-509294059>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJXHQZZDRFLURK3KQ4XD5ULP6NSLXANCNFSM4H5H5UNA> .

[jywarren]

well, can you list out the kinds of selections that need to be made, so we can think about what associated data would have to go with a .csv file? Thanks!

…

On Mon, Jul 8, 2019 at 1:07 PM Isha Gupta ***@***.***> wrote: Yeah, we will create or use the current function for plotting the graph, which already has the required selections, so I believe that should work? On Mon, Jul 8, 2019, 9:50 PM Jeffrey Warren ***@***.***> wrote: > Oh, i see. So the configuration of SDG would need to be added to the > short-code perhaps? How would that work? What are the parameters that would > need to be stored based on the selection of columns and such? > > On Mon, Jul 8, 2019 at 12:14 PM Isha Gupta ***@***.***> > wrote: > > > That sounds pretty good, although a little tough, since we need to run > the > > entire thing for the selection of columns and stuff in the drag drop box > > itself. What I suggest is, we create a "Publish as research note" button > > where the graphs are plotted, and from there itself it can pick up the > > plotted graphs and proceed in a similar manner as above. What do you > think? > > > > On Mon, Jul 8, 2019, 9:34 PM Jeffrey Warren ***@***.***> > > wrote: > > > > > I think we could start by implementing the [simple-graph:____.csv] > > > filtering in wikis/notes, without also doing drag/drop at the same > time, > > as > > > a first step. We could for example, as a first step, make it so you > have > > to > > > just drag/drop a CSV as you do now, but then change [graph:____.csv] to > > > [simple-graph:____.csv] and then your code would start working. Then > > > later we could decide how we want to handle the dragging and dropping! > > > How's that sound? > > > > > > — > > > You are receiving this because you were mentioned. > > > Reply to this email directly, view it on GitHub > > > < > > > #65?email_source=notifications&email_token=AJXHQZZQHYSGPQ2GDIG44HTP6NQRTA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNR7WY#issuecomment-509288411 > > >, > > > or mute the thread > > > < > > > https://github.com/notifications/unsubscribe-auth/AJXHQZYGNLBLGTSDH7PQ2MTP6NQRTANCNFSM4H5H5UNA > > > > > > . > > > > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > < > #65?email_source=notifications&email_token=AAAF6JZ2DMB4XJ6FHBIXBWDP6NRUXA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNS25Y#issuecomment-509291895 > >, > > or mute the thread > > < > https://github.com/notifications/unsubscribe-auth/AAAF6J2ILZWSFCJVP6XGVTDP6NRUXANCNFSM4H5H5UNA > > > > . > > > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < #65?email_source=notifications&email_token=AJXHQZ6LJZHOYZEZV7PO4L3P6NSLXA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNTL2Y#issuecomment-509294059 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AJXHQZZDRFLURK3KQ4XD5ULP6NSLXANCNFSM4H5H5UNA > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#65?email_source=notifications&email_token=AAAF6J6N3ULJT6AG45T4HJLP6NX3LA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNXPVQ#issuecomment-509310934>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAF6J3XVG2OMBEESJT437DP6NX3LANCNFSM4H5H5UNA> .

[IshaGupta18]

Yes, just let me study about the short-code a little more, then I'll get back to you with the parameters. If I am not mistaken, we would be just needing the plotgraph function, which already has the selected parameters, that should do the trick. Thanks a lot! On Tue, Jul 9, 2019, 12:21 AM Jeffrey Warren <notifications@github.com> wrote:

…

well, can you list out the kinds of selections that need to be made, so we can think about what associated data would have to go with a .csv file? Thanks! On Mon, Jul 8, 2019 at 1:07 PM Isha Gupta ***@***.***> wrote: > Yeah, we will create or use the current function for plotting the graph, > which already has the required selections, so I believe that should work? > > On Mon, Jul 8, 2019, 9:50 PM Jeffrey Warren ***@***.***> > wrote: > > > Oh, i see. So the configuration of SDG would need to be added to the > > short-code perhaps? How would that work? What are the parameters that > would > > need to be stored based on the selection of columns and such? > > > > On Mon, Jul 8, 2019 at 12:14 PM Isha Gupta ***@***.***> > > wrote: > > > > > That sounds pretty good, although a little tough, since we need to run > > the > > > entire thing for the selection of columns and stuff in the drag drop > box > > > itself. What I suggest is, we create a "Publish as research note" > button > > > where the graphs are plotted, and from there itself it can pick up the > > > plotted graphs and proceed in a similar manner as above. What do you > > think? > > > > > > On Mon, Jul 8, 2019, 9:34 PM Jeffrey Warren < ***@***.***> > > > wrote: > > > > > > > I think we could start by implementing the [simple-graph:____.csv] > > > > filtering in wikis/notes, without also doing drag/drop at the same > > time, > > > as > > > > a first step. We could for example, as a first step, make it so you > > have > > > to > > > > just drag/drop a CSV as you do now, but then change [graph:____.csv] > to > > > > [simple-graph:____.csv] and then your code would start working. Then > > > > later we could decide how we want to handle the dragging and > dropping! > > > > How's that sound? > > > > > > > > — > > > > You are receiving this because you were mentioned. > > > > Reply to this email directly, view it on GitHub > > > > < > > > > > > #65?email_source=notifications&email_token=AJXHQZZQHYSGPQ2GDIG44HTP6NQRTA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNR7WY#issuecomment-509288411 > > > >, > > > > or mute the thread > > > > < > > > > > > https://github.com/notifications/unsubscribe-auth/AJXHQZYGNLBLGTSDH7PQ2MTP6NQRTANCNFSM4H5H5UNA > > > > > > > > . > > > > > > > > > > — > > > You are receiving this because you were mentioned. > > > Reply to this email directly, view it on GitHub > > > < > > > #65?email_source=notifications&email_token=AAAF6JZ2DMB4XJ6FHBIXBWDP6NRUXA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNS25Y#issuecomment-509291895 > > >, > > > or mute the thread > > > < > > > https://github.com/notifications/unsubscribe-auth/AAAF6J2ILZWSFCJVP6XGVTDP6NRUXANCNFSM4H5H5UNA > > > > > > . > > > > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > < > #65?email_source=notifications&email_token=AJXHQZ6LJZHOYZEZV7PO4L3P6NSLXA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNTL2Y#issuecomment-509294059 > >, > > or mute the thread > > < > https://github.com/notifications/unsubscribe-auth/AJXHQZZDRFLURK3KQ4XD5ULP6NSLXANCNFSM4H5H5UNA > > > > . > > > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < #65?email_source=notifications&email_token=AAAF6J6N3ULJT6AG45T4HJLP6NX3LA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNXPVQ#issuecomment-509310934 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AAAF6J3XVG2OMBEESJT437DP6NX3LANCNFSM4H5H5UNA > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#65?email_source=notifications&email_token=AJXHQZ7ZFZ26ICKEQHXLW4DP6OECXA5CNFSM4H5H5UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZOANOY#issuecomment-509347515>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJXHQZ5EU62OOJ62EBGT2C3P6OECXANCNFSM4H5H5UNA> .

[IshaGupta18]

Okay, so I have been sleuthing through all the codes and files that come together and create the inline graph feature and I found a lot of interesting stuff, much of which I understood, but I still have some doubts one of them being, from where exactly are we sending this post request to /images

And in my study I found the entire trace: we go from show.html.erb to application_helper to node_shared which calls the partial _graph.html.erb which calls a functions in graph.js and presto! we get our inline graph.
But what I am not getting is this post request. Is it coming from drag_drop.js or where? Because I see this

when I drag and drop a CSV, so where is this coming from exactly. I would be following a similar track to achieve this new feature, so thats why I need so much clarity. Please let me know the last bits and pieces of this puzzle!

@jywarren @gauravano @Souravirus @namangupta01

[IshaGupta18]

There are some links between this flow that I have written which are missing. For example where exactly is this tag created with a particular name like this and this format of values?