UNHANDLED REJECTION VipsJpeg: Premature end of JPEG file

[smoke-indica]

I'm getting the following error which halts the gatsby develop command:

success createPages - 1.524s            
                                                                                                                                                                           [                            ]   215.425 s 20/3145 1% run queries                                                                                                                                                  

[=====================       ]   215.060 s 133/178 75% Downloading remote files                                                                                                                                    

[==========                  ]   6.619 s 11/29 38% Generating image thumbnails                                                                                                                                                                                                                                                                                                                                                         

ERROR                                                                                                                                                                                                                                                                                                                                                                                                                                

UNHANDLED REJECTION VipsJpeg: Premature end of JPEG file                                                                                                                                                           

VipsJpeg: out of order read at line 144                                                                                                                                                                            

VipsJpeg: Premature end of JPEG file                                                                                                                                                                               

VipsJpeg: out of order read at line 144

Rather than fail on error, would it be possible to replace with a placeholder image?

Thanks

edit: Found some possibly related issues:

lovell/sharp#1859

gatsbyjs/gatsby#13442

edit: added try/catch to several await functions in this repo & found the following issue was underlying:

gatsbyjs/gatsby#6291

[d4rekanguok]

Thank you for opening an issue & digging in!

It makes sense to ignore image transformation failure during development.

Does this only happen in development for you, or in production/ci as well? A few comments in the linked issues indicate that a clean slate 'gatsby build/develop' may fix the problem. I was thinking of using a placeholder image in development to avoid wasting time/bandwidth when Gatsby reset its cache — that could help with this as well.

[smoke-indica]

Just reinstalled the node modules, develop still fails, tried build and it gets further but eventually fails in a similar manner:

 ERROR #85901  GRAPHQL                                                                                                                                                                                                                                                                                                                                                                                                                There was an error in your GraphQL query:                                                                                                                                                                                                                                                                                                                                                                                             VipsJpeg: Premature end of JPEG file  

not finished run queries - 1125.855s                                                                                                                                                                               not finished Downloading remote files - 1089.544s                                                                                                                                                                  not finished Generating image thumbnails - 893.607s 

Build did output the following messages which weren't output in development:

warn failed to download                                                                                                                                                                                            warn failed to download                                                                                                                                                                                            warn failed to download

Placeholders in development, or avoiding downloading identical images would be cool.

[d4rekanguok]

Interesting, that failed to download line is definitely from this plugin:

gatsby-remark-images-anywhere/src/util-download-image.ts

Lines 33 to 57 in 2193436

	if (!imageFileNode) {
	try {
	const imageUrl = process.env.LOW_WIFI_MODE
	? 'https://placekitten.com/1200/800'
	: url
	const fileNode = await createRemoteFileNode({
	url: imageUrl,
	store,
	cache,
	createNode,
	createNodeId,
	reporter,
	parentNodeId: id,
	})
	if (fileNode) {
	imageFileNode = fileNode
	await cache.set(mediaDataCacheKey, {
	fileNodeId: fileNode.id,
	})
	}
	} catch (e) {
	reporter.warn(`failed to download ${url}`)
	}
	}

If there's no URL next to that warning, it means I might have done something wrong at some point and interpret something as an url even though it shouldn't had been. Is it possible to pinpoint which url is causing the error? Sorry about the hassle!

Also I didn't realized that I have already coded in a dumb placeholder (cat image) for not downloading image during development. You can enable it by setting LOW_WIFI_MODE:

LOW_WIFI_MODE=true gatsby develop

I'll put in some work to make the placeholder more informative (show image size, etc) and customizable.

[smoke-indica]

After updating to the latest version of sharp I no longer get these UNHANDLED REJECTION VipsJpeg: Premature end of JPEG file warnings anymore.

I left it to run overnight and it halted on the last couple images:

success Generating image thumbnails - 21555.921s - 31350/31350 1.45/s
[============================]   21754.901 s 3256/3259 100% run queries
[=========================== ]   21754.510 s 7622/7668 99% Downloading remote files

The message success createPages - 1.058s occurs when it downloads an image rather than say success downloading image - 1.058s, it emits the log when 32.116 s 20/43 47% Downloading remote files increments.

7hrs is too long, I've got to optimize the settings.. lol

            options: {
              maxWidth: 960,
              quality: 75,
              wrapperStyle: 'margin-bottom: 0.5rem;',
              loading: 'lazy',
              sharpMethod: 'fluid',
              useMozJpeg: true,
              toFormat: 'WEBP'
            },

After canceling the task, running gatsby develop ignores the previously downloaded files and starts from scratch.

Would it not be faster to generate all the download links & download them all at once?

[d4rekanguok]

Wow, sorry to hear it still take so long — It's strange that running gatsby develop ignore the previously downloaded files... Unless node_modules & .cache is wiped the downloaded images should still be there. If this doesn't happen then it must have been a bug, either in our plugin or on Gatsby's side. I could investigate if you have a reproduction somewhere...

Would it not be faster to generate all the download links & download them all at once?

I thought that's how gatsby work already, collecting all remote links & download them at once? I may misremember though