1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
|
Trial Inference API Extension Example
=====================================
About the API
:::::::::::::
The trial inference API is an experimental feature that allows you to run inference tasks in your web extension.
It's wrapping our internal inference API, see `its documentation <https://firefox-source-docs.mozilla.org/toolkit/components/ml/api.html>`_.
endpoints:
- `browser.trial.ml.createEngine`: creates an inference engine.
- `browser.trial.ml.runEngine`: runs an inference engine.
- `browser.trial.ml.onProgress.addListener`: add a listener for engine events.
The pattern to use the inference API is:
.. code-block:: js
// Initialize the event listener
browser.trial.ml.onProgress.addListener(progressData => {
console.log(progressData);
});
// Create the engine, may trigger downloads.
await browser.trial.ml.createEngine({
modelHub: "huggingface",
taskName: "summarization",
});
// Call the engine
const res = await browser.trial.ml.runEngine({
args: ["This is the text to summarize"],
});
// Get the results.
console.log(res[0]["summary_text"]);
About the example
:::::::::::::::::
This web extension implements an alt text feature on any image in the browser.
You can right click on the image and you will get an alt text generated by
an `image-to-text` model.
Pre-requisites
--------------
If not running in Nightly, make sure your browser has the inference engine and the experiments
enabled by switching the following flags in `about:config`:
- `browser.ml.enable`: true
- `extensions.experiments.enabled`: true
Installation
------------
1. Go into `about:debugging` > `This Nightly` > `Load Temporary Add-on`
2. Go into the directory containing the extension (`toolkit/components/ml/docs/extensions-api-example`).
3. Pick `manifest.json` and load it.
4. Once the extension is loaded, go into `about:addons` and enable the optional permission.
Usage
-----
In a web page with images, right-click on an image then chose "Generate ML description".
The first time you will use it, the model will be downloaded from our servers and
you will get a download progress.
|
