Criteria used to assess opennness of projects/tools/platforms?

[xolotl]

From @bmkramer on May 9, 2018 8:17

Has it already been decided what criteria will be used to assess openness of projects/tools/platforms to be included in the roadmap? (and perhaps also to identify areas where improvement is recommended)?

For instance, the [Principles of Open Scholarly Infrastructure] (http://dx.doi.org/10.6084/m9.figshare.1314859) might be useful for assessing the infrastructure of tools and platforms. Perhaps in combination with aspects of the FORCE11 principles of the scholarly commons (see also preprint) for both the openness of resulting research objects (Open, FAIR and citable) and openness to participation.

Full disclose: this ties in closely to some of the work we are ourselves planning to do over the next months*, and it would be great to see if we can collaborate!

*See the proposals for FORCE2018 that @JeroenBosman and I put in:

• On the positive side: workflows with commons-compliant tools and platforms
• If you build it (open), do they come?

Copied from original issue: OpenScienceRoadmap/mozilla-sprint-2018#13

[xolotl]

From @Daniel-Mietchen on May 9, 2018 13:58

@bmkramer I'm certainly looking forward to interaction along these lines!

[xolotl]

From @HeidiSeibold on May 10, 2018 9:10

I think this is a very important question which we should probably discuss very early on. For me, important aspects are:

• Open source and with an open license. If the basis of the project isn't open it's not particularly good for open practices. This is maybe a bit strict.
• Freely available (at least to those who are unable to pay the cost). In a stricter sense this may include not having to "pay" with personal information or having to look at ads.
• FAIR. I feel like the FAIR principles can be valuable for tools. The also need to be Findable, Accessible, Interoperable, and Reusable.

[xolotl]

From @Daniel-Mietchen on May 10, 2018 15:50

@HeidiSeibold
Those are good starting points. Some things I'd add would be the Open Definition as well as some measures for transitioning from or interfacing with non-open tools or infrastructure.

The question of defining what is meant by "open" also popped up at drdvo/OWLTEH#8 , and there are related questions on Ask Open Science (pinging our issue #6 ):

• What does "open" mean in science?
• How is open science defined?

[xolotl]

From @drdvo on May 10, 2018 16:18

You might also find useful the 5R permissions, in relation to the notion of 'Open Educational Resources' (OER) but relevant more generally to definitions of open content.

[xolotl]

From @Daniel-Mietchen on May 10, 2018 17:14

@bmkramer In a chat around our Mozsprint, @felliott just brought up your graph at https://user-images.githubusercontent.com/16495178/39816678-e8c85458-5369-11e8-9e09-b09ad6b1fae8.png . Do you have an editable version of that, so we could go and try to filter that by whatever openness criteria we come up with?

[xolotl]

From @bmkramer on May 10, 2018 19:0

Yes, I have, it was made in powerpoint (ahem) - I'll look it up on my harddrive and figure out how to upload/share in a moment.

Some background on what is currently on that figure (and what isn't), see also here:

The logo's were the tools and platforms (not necessarily all open!) that we asked about in our 2015-2016 survey on tool/platform usage, ordered per the different phases of the research workflow that we also used in the survey.

The grey lines are tools/platforms that people had indicated using together, either within or between 'adjacent' workflow phases. This co-use represented raw data, later on we did some more thorough analysis to identify tools/platforms preferentially used together - those results are shown here in an interactive table.

Finally, the green highlight depicts a hypothethical workflow formed by tools and platforms across the workflow that had been identified as mostly compliant with the (then) version of the principles of the scholarly commons, during a workshop held in San Diego in Sept 2016. (@Daniel-Mietchen - you may remember :-)

Enough talk, off to search my harddrive!

[edited to add: this is taking a bit longer, apologies. Hope to have it for you by tomorrow, to use on the second sprint day. The first day is well and truly over in my timezone....]

[xolotl]

From @bmkramer on May 11, 2018 11:28

OK, here's the editable powerpoint. Contrary to what you might hope/expect, the circles surrounding the tools together with the grey lines connecting them are a static image, because it was generated as a network image in Gephi (based on our survey results).

101innovations_workflow_puzzle.pptx

An additional suggestion, if you will be scoring tools along openness criteria, it could be useful to add that info to our database of 400+ tools/platforms, e.g. in column L in the 'Data' tab. (we are behind in updating this list, so feel free to add tools suggestions)
http://bit.ly/innoscholcomm-list

Hope this is helpful!

[xolotl]

Just a note to tie this issue together with two others that are focusing on visualizing the open science tool ecosystem (#11) and locating data about it in wikidata (#7).

[xolotl]

From @Daniel-Mietchen on May 11, 2018 15:52

I've just browsed the spreadsheet for a while and think we could do a number of things with it:

• add some columns in the Data sheet to cover some dimensions of openness, e.g.
  • open source
  • code location
  • code license
  • open data
  • data location
  • data license
• add a row (perhaps initially just under the metadata rows 2 and 3) with the corresponding Wikidata properties
• add a column (perhaps initially to the right of column B) with the Wikidata item for the respective tool or website
• translate concepts in the free-form text (e.g. in columns F, H or J) once into the corresponding Wikidata items

On that basis, we could then filter for those that meet any of the openness criteria, convert the table into a format compatible with QuickStatements, and edit/ create the corresponding Wikidata entries.

[xolotl]

From @felliott on May 11, 2018 15:57

I'll start looking at converting it to QuickStatements.