dioxus barcode scanner

[brianmay]

Hello,

Crazy idea maybe, but I would like a wasm front end written in dioxus to be able to prompt the user to scan a barcode.

Not sure where to start... Or if this is feasible.

I guess there are two distinct steps:

1. grab photo using camera.
2. detect any barcodes in that photo.

For step 2, It looks like there is on official detection API, https://developer.mozilla.org/en-US/docs/Web/API/Barcode_Detection_API

But by the looks of it browser support might be limited. Not sure if Mobile firefox on Android supports it, might be limited to Chrome on Android.

Then somebody has written a component for yew: https://gitlab.com/voltfang-public/yew-scanner

Code here: https://gitlab.com/voltfang-public/yew-scanner/-/blob/main/src/lib.rs?ref_type=heads

It looks like this grabs an image from a HtmlVideoElement and passes to the bardecoder crate for decoding (not sure I entirely understand how the HtmlVideoElement gets its image.

https://github.com/piderman314/bardecoder

But it does certain things which I am not sure of in Dioxus, e.g. grabbing references to rendered html elements.

Then there is some JavaScript code:

https://stackoverflow.com/questions/67078359/barcode-scanning-via-mobile-browser

And I see this library looks like it might have Camera support in the future, but isn't quite there yet:

https://github.com/DioxusLabs/sdk

Any suggestions on which approach I should take?

[alkeryn]

i figured this discussion may be helpful, you can call native api's.

[brianmay]

I am unclear what you intended here. Did you mean to add a link?

[alkeryn]

Oh yea sorry, i was tired and didn't realized it didn't paste
Here is the link. #3475

My current issue rn is more how to figure background tasks on mobile with dioxus.

[brianmay]

I had a fiddle with some code proof of concept code. https://stackoverflow.com/a/78089783, which uses the barcode scanner API. This has poor browser support, but there is a polyfill for it.

Amazingly it seems to work, at least on Desktop with Camera. Proof of concept code only though.

I can't think right now, but wondering:

1. Is there some way to npm install the polyfill package (see https://github.com/undecaf/barcode-detector-polyfill) and use that from within dioxus?
2. How do I return the scanned barcode from JavaScript to the wasm code? Ideally I would like to be able to set a signal. Somehow I suspect it may not be that simple. Particularly as the Javascript code is in a loop. I guess I could exit as soon as a barcode is found though maybe.

#[component]
fn Barcode() -> Element {
    use_effect(move || {
        document::eval(
            r#"
            console.log("00000");

            //WebAssembly polyfill for some browsers
            try { window['BarcodeDetector'].getSupportedFormats() }
            catch { window['BarcodeDetector'] = barcodeDetectorPolyfill.BarcodeDetectorPolyfill };
            console.log("1111");

            // Define video as the video element. You can pass the entire element to the barcode detector!
            const video = document.querySelector('video');
            console.log("2222");

            // Get a stream for the rear camera, else the front (or side?) camera, and show it in the video element.
            video.srcObject = await navigator.mediaDevices.getUserMedia({ video: { facingMode: 'environment' } });
            console.log("33333");

            // Create a BarcodeDetector for simple retail operations.
            const barcodeDetector = new BarcodeDetector({ formats: ["ean_13", "ean_8", "upc_a", "upc_e"] });
            console.log("4444");

            while(true) {
                try {
                    // Try to detect barcodes in the current video frame.
                    let barcodes = await barcodeDetector.detect(video);

                    // Continue loop if no barcode was found.
                    if (barcodes.length == 0)
                    {
                        // Scan interval 50 ms like in other barcode scanner demos.
                        // The higher the interval the longer the battery lasts.
                        await new Promise(r => setTimeout(r, 50));
                        continue;
                    }

                    console.log("xxxx", barcodes);

                    // We expect a single barcode.
                    // It's possible to compare X/Y coordinates to get the center-most one.
                    // One can also do "preferred symbology" logic here.
                    // document.getElementById("barcode").innerText = barcodes[0].rawValue;

                    // Notify user that a barcode has been found.
                    // navigator.vibrate(200);

                    // Give the user time to find another product to scan
                    await new Promise(r => setTimeout(r, 1000));
                }
                catch(err) {
                    console.log("failed", err);
                    //Wait till video is ready
                    //barcodeDetector.detect(video) might fail the first time
                    await new Promise(r => setTimeout(r, 200));
                }
            }
        "#,
        );
    });

    rsx! {
        script { src: "https://cdn.jsdelivr.net/npm/@undecaf/zbar-wasm@0.9.15/dist/index.js" }
        script { src: "https://cdn.jsdelivr.net/npm/@undecaf/barcode-detector-polyfill@0.9.21/dist/index.js" }
        h1 { "Penguins Rule"}
        video { autoplay: true, playsinline: true,  }
    }
}

[brianmay]

I managed to integrate this into my app at https://github.com/brianmay/penguin_nurse/

I used https://www.youtube.com/watch?v=ZF0IPdSVbPw for inspiration.

But I did not use esbuild, as it seemed to have problems building the library I needed. See undecaf/barcode-detector-polyfill#20

Instead I used rollup. And the the rollup.config.mjs from https://github.com/undecaf/barcode-detector-polyfill/tree/master/example-bundled which renames some references inside the code. Yes, this seems yuck. But this is required if you want to avoid using an external copy of zbar.wasm. Which isn't a huge thing, apart from a large comment it is already minified by rollup anyway.

I couldn't use this output in the document::eval, it had import statements which failed. And I couldn't work out how to configure rollup to stop this.

So I used in my top level page.

    rsx! {
        document::Script {
            type: "module",
            src: asset!("/assets/bundle.js", JsAssetOptions::new().with_minify(false)),
        }
    ...
    }

Where bundle just loads the polyfill to window['BarcodeDetector'], and does nothing else.

Then in my document::eval I can use the already loaded polyfill as before, with inline javascript code. https://github.com/brianmay/penguin_nurse/blob/bfeb630f18f862b645f7068e578d78cb11eb2aef/src/forms/barcodes.rs#L8-L65

I also had problems with the js minification of the file creating an invalid import line. As it created an import line with the absolute filesystem path, which Firefox complained loudly. So (as above) I disabled that.

Uncaught (in promise) TypeError: URL constructor: /home/brian/tree/personal/penguin_nurse/assets/bundle.js is not a valid URL.

I think this was caused by the minified file containing:

const importMeta={url:"/home/brian/tree/personal/penguin_nurse/assets/bundle.js" ...

Wondering if this is a bug.

Then I had the problem then that barcode-detector-polyfill really wants to be able to load /assets/zbar.wasm

I can include it using:

let _zbar = asset!("/assets/zbar.wasm");

but dioxus wants to serve it as /assets/zbar-6a921bd2bb9606f1.wasm, and I couldn't work out how to do this with barcode-detector-polyfill.

So I had to rename that inside the js also: https://github.com/brianmay/penguin_nurse/blob/bfeb630f18f862b645f7068e578d78cb11eb2aef/rollup.config.mjs#L24

Not sure I like this, it will break if the file changes. But best I can think of right now.