Not able to search by inventory number

[zsofiaj]

In our Italian Paintings publication, we are not able to search by inventory number. For example, for this object, if I search the site for 1871.6, the inv. no., there are no results and the screen just stays blank. It does not say anything like "no results." @Erin-Cecele could this be a bug? It is really important for most of our projects to be able to search by inv. no. Thank you!!

https://italian-paintings-live.netlify.app/catalogue/volume-1/the-florentine-school/9/

Another (unrelated) issue with the search is that the italics are not setting properly, as seen in the screenshot below.

[Erin-Cecele]

Hi @zsofia, This is similar to a question asked by @salger1 and @mgray4. They noted that their search was not pulling in tombstone information. This is because search is not including object.yaml information. We are testing out a possible workaround for this. Stay tuned.

[zsofiaj]

Thanks so much, @Erin-Cecele . Any ideas for a workaround would be greatly appreciated. It seems pretty important to be able to search the data in the objects.yaml file since readers are likely to search by accession number, especially since we don't have catalogue numbers visible to readers.

Would tombstone data show up in the search if I had placed object data directly in the page YAML, rather than keeping it all in the objects.yaml file? I'm hoping I won't need to make this change for our Italian Paintings book at this late stage, but if this would solve the search problem, I'll definitely go this route for our Indonesian Textiles publication.

[Erin-Cecele]

I believe Quire won't search anything in the YAML format regardless of whether it's on the entry Markdown page or in a .yaml file, but @geealbers could provide more accurate information.

[zsofiaj]

@Erin-Cecele I just want to add a note here to say that I've been discussing this with @1000camels , and Darcy suggested the below modifications to the _plugins/search/search.js and _plugins/search/write.js files. At first, this solution didn't work for us, because I guess the search is only indexing the content files, and not the data files. Darcy suggested adding the "inv_no" to the page yaml for each entry, and that worked. So now I have the inv no for each object both in the objects.yaml and on each entry page. Can you foresee any issues that might cause?

If you'd like to test this on our preview site, you could try searching by inv. no. for one of the first ten entries, for example, this one:
https://italian-paintings-preview.netlify.app/catalogue/volume-1/the-florentine-school/2/

I haven't updated the rest yet. Here are the other changes we made.

We edited _plugins/search/search.js and added inv_no to the list of what is returned from buildIndex:

import lunr from 'lunr'

/**
* A full-text search module that implements methods to build and query an index of content.
*
* @class Search
*/
export default class Search {
constructor(data) {
this.contentList = data
this.index = this.buildIndex(data)
}

buildIndex(pages) {
return lunr(function () {
this.field('abstract')
this.field('content')
this.field('contributor')
this.field('subtitle')
this.field('title')
this.field(inv_no)
this.field('url')
this.ref('url')
pages.forEach((page) => {
this.add(page)
})
})
}

/**
* Query the index of for the provided string or terms
*
* @param {String} query Terms for which to search
* @return {Array} An array of result objects
*/
search(query) {
const results = this.index.search(query)
return results.map((result) => this.contentList.find(({ url }) => url === result.ref))
}
}

And we edited _plugins/search/write.js and add inv_no to the list of properties pulled from data and what is returned.

const chalkFactory = require('~lib/chalk')
const fs = require('fs-extra')
const path = require('path')
const logger = chalkFactory('Search Index')
/**
* Write JSON index of page content in `collections.html`
*
* @param {Object} collections
* @return {Object} Page content index JSON
*/
module.exports = function (collections) {
const wordcount = (content) => {
if (!content) return 0
return content.split(' ').length
}

const contentIndex = collections.html.map(({ data, templateContent }) => {
const { abstract, canonicalURL, contributor, subtitle, title, layout, inv_no } = data
return {
abstract,
content: templateContent,
contributor,
length: wordcount(templateContent),
subtitle,
title,
type: layout,
url: canonicalURL,
inv_no,
}
})

const outputDir = process.env.ELEVENTY_ENV === 'production' ? 'public' : '_site'
const outputPath = path.join(outputDir, 'search-index.json')

try {
fs.ensureDirSync(path.parse(outputPath).dir)
fs.writeJsonSync(outputPath, contentIndex)
} catch (error) {
logger.error(error)
}
}

[zsofiaj]

And THANK YOU, again, to @1000camels !!

[1000camels]

I would like to load object properties in order to index objects by id, but wasn't able to figure out how to do it in the time I spent on it. I may return to this. It seems like it would be great if search properties could be defined in config.yaml and would allow you to specify object.id.inv_no as a potential indexible property.

If anyone can help me get access to objects, I will continue to work on it. I tried to use getObject(object.id), but it was not available.

[1000camels]

I have managed to access getObject, but for some reason I cannot access the object id in the front-matter. I would love a developer to give me some tips on what is inside collections and how I might go about access object id so I can get the data from objects.yaml

[cbutcosk]

@1000camels Some helpful context is that collections is an 11ty Collections API object, so what you're getting back is a page template object with headmatter / templating vars / etc in the data property:

  eleventyConfig.on('eleventy.after', () => {
    collections.html.forEach( ({ data }) => { // `item` in 11ty's docs
      const { object } = data
      const { id, inv_no } = object 

      // Do something cool with `id` or `inv_no` -- get the data from objects.yaml, etc
    })    
  })

getObject for its part mostly IIRC just shovel deserialized YAML out of objects.yaml so I believe any custom props you add to the objects list should come across that interface faithfully.

[1000camels]

Thanks @cbutcosk I am not sure where this code should live (in eleventy.js before calling collectionsPlugin() perhaps?)

I am still learning how to think about this environment, but I can imagine how expanding collections['object'] to include the properties of objects.yaml in the Collections API object would be beneficial all around. And if it was included, I could definitely use in the search index. I might also be able to figure out a way to make those search index terms a setting, which I think would be super useful for people.

I would love more help from Quire developers to understand how to work on this code. Any tips on developing and testing would be useful as well. I am mostly using quire preview which seems overkill. Might be nice to understand what underlying commands I can use as well as debugging flags to try to understand how to test a particular piece of code before including it the a project.

[cbutcosk]

@1000camels One clarification -- according to their docs in the 11ty pages collection API collections.object would be an array of pages in the publication with a tags headmatter property including - object or created via eleventyConfig.addCollection(). In the snippet above collections.html is a special collection that quire uses to get all the pages in the HTML representation, with similar page collections under collections.epub and collections.pdf. In addition to this grouping mechanism(s) collections is useful when a piece of code needs to access page content without triggering 11ty's premature page content error.

The snippet that Sofia posted from _plugins/search/write.js to open this thread uses collections.html to walk all HTML pages in the publication in order to read the page data and then serialize the page data output to a place where it'll be transferred with site content. So if you want to add inv_no to search and then use it you would modify the code I put above into that plugin and make the corresponding changes to _plugins/search/search.js.

Re: Dev experience, npm run debug and npm run build are both useful as are --debug flags to quire commands where available (most quire commands also take a --help flag to show any other verbosity / log level flagging for a command).

Tests in quire are an evolving story but there should be some example built HTML / CSS tests soon that can at least guide implementers in creating tests for their custom integrations.

[1000camels]

Thanks for the dev tips. I managed to find npm run debug. I also found advice on adding some break points to be able to check the state. I was struggling to understand why I could not get id from object in the page headmatter. I was tripping over the fact that not all pages have object and id defined in the headmatter.

The below code works.

[1000camels]

I found a way to add the object properties to the collections API. I don't really have a sense of any performance issues with this, but if they are negligible, I would think it would be useful to add this to quire:

Added to eleventy.js, line 118

/**

• Adding object properties from objects.yaml to collections, so they can be indexed in search
  */
  eleventyConfig.on("eleventy.after", () => {
  const getObject = eleventyConfig.getFilter("getObject");
  collections.html.forEach(({ data }) => {
  // item in 11ty's docs
  const { object } = data;
  if (object) {
  const id = object[0]?.id || {};
  // Do something cool with id or inv_no -- get the data from objects.yaml, etc
  const this_object = id !== "undefined" ? getObject(id) : null;
  data["object"] = this_object;
  }
  });
  });

I am also going to look for a way to convert the search terms into configurable properties, as ultimately, we only wanted to access one property

[Erin-Cecele]

@fchasen I'm including you in this thread because you'll be working on improving the search function for Getty's Quire publications. Ideally, we'll be able to integrate improvements to the search function into core Quire.