AmCharts to display Data Provider information - (Custom Javascript in Embed Component)

[SufiSR]

Disclaimer

I am a user of Budibase, not a maintainer.
This article follows the discussion #6839

The challenge

We are using Budibase internally not only for forms and workflows but also to gather information from multiple sources and produce and display statistics and KPIs. While Budibase provides some Charts most of them are rather "static" and we have run into various challenges with them (e.g. no second X Axis). We have used AmCharts in the past as it has a vast catalogue of charts as well as very flexible displaying options.

The Solution

Based on the other article it is possible to inject javascript code using the example:

<img style="display: none;" src onerror="
	// JavaScript goes here!
	alert('Hello world!');
	// Be careful to escape quotation marks or using &quot; or something like single quotes,
	// the onerror attribute is surrounded by double quotes in this example.
"> 

I do understand that this is not too pretty but it does solve my problem (as I am personally not able to create my own components (too much knowledge missing - I am not a developer after all). Plus, I believe that a charting library like AmCharts or HighCharts as a component would be a lot of work.

Injecting the external scripts

AmCharts expects it's own scripts to be in the DOM. I failed with various approaches injecting the scripts. They would show, but they wouldn't be properly loaded (or not loaded on time). Eventually, I managed to properly load the scripts with this code. Please note that you need to make sure to work with a specific set of quotation marks.

// We want the scripts to be added to the body
var parentNode = document.querySelectorAll(`[data-name='New Embed']`);

function loadScript(url, callback) {
  var script = document.createElement('script');
  script.type = 'text/javascript';
  script.src = url;
  script.onload = callback;
  document.head.appendChild(script);
}

// Define the URLs of the amCharts script files
var amChartsLibURL = 'https://cdn.amcharts.com/lib/5/index.js';
var amChartsXYURL = 'https://cdn.amcharts.com/lib/5/xy.js';
var amChartsThemeURL = 'https://cdn.amcharts.com/lib/5/themes/Animated.js';

// Load the amCharts scripts in sequence
loadScript(amChartsLibURL, function() {
  loadScript(amChartsXYURL, function() {
    loadScript(amChartsThemeURL, function() {
      // All amCharts scripts are loaded
      // You can now initialize your amCharts chart

Next step was to add the generic code for charts that is used by AMCharts for the configuration as found in their documentation

Grabbing the data from a Data Provider

I did not manage to get the data from the data provider directly into the script (neither the $-Javascript style would work nor the {{ Data Provider.Rows }} style.

Therefore, I decided to use another embed (also better for using it with other data) where I would run the transformations and then just add it as non-displayed text within a new div in the DOM:

I used the Budibase Sample Database Jobs Table for this example:

const data = $("Sample Jobs Data.Rows")
var transformedData = data.map(function(item) {
  return {
    CustomerName: item['Customer Name'],
    QuotePrice: item['Quote Price']
  };
});

return '<div  data-name="raw-data-custom2" style="display: none">'+JSON.stringify(transformedData)+'</div>'

This then allows me to retrieve the data in my injection script:

//get data
const element = document.querySelector(`[data-name='raw-data-custom2']`);
const text = element.textContent;
var data = JSON.parse(text);

And here is the result:

Full Code for Injection (Second Embed)

<img style="display : none;" src onerror="

var parentNode = document.querySelectorAll(`[data-name='Data Transformation']`);

function loadScript(url, callback) {
  var script = document.createElement('script');
  script.type = 'text/javascript';
  script.src = url;
  script.onload = callback;
  document.head.appendChild(script);
}

// Define the URLs of the amCharts script files
var amChartsLibURL = 'https://cdn.amcharts.com/lib/5/index.js';
var amChartsXYURL = 'https://cdn.amcharts.com/lib/5/xy.js';
var amChartsThemeURL = 'https://cdn.amcharts.com/lib/5/themes/Animated.js';

// Load the amCharts scripts in sequence
loadScript(amChartsLibURL, function() {
  loadScript(amChartsXYURL, function() {
    loadScript(amChartsThemeURL, function() {
      // All amCharts scripts are loaded
      // You can now initialize your amCharts chart


// Create a div element for the chart
var chartDiv = document.createElement('div');
chartDiv.id = 'chartdiv';
chartDiv.style.width = '100%';
chartDiv.style.height = '500px';
parentNode[0].appendChild(chartDiv);



// Create root element
// https://www.amcharts.com/docs/v5/getting-started/#Root_element
var root = am5.Root.new('chartdiv');


// Set themes
// https://www.amcharts.com/docs/v5/concepts/themes/
root.setThemes([
  am5themes_Animated.new(root)
]);


// Create chart
// https://www.amcharts.com/docs/v5/charts/xy-chart/
var chart = root.container.children.push(am5xy.XYChart.new(root, {
  panX: true,
  panY: true,
  wheelX: 'panX',
  wheelY: 'zoomX',
  pinchZoomX: true
}));

// Add cursor
// https://www.amcharts.com/docs/v5/charts/xy-chart/cursor/
var cursor = chart.set('cursor', am5xy.XYCursor.new(root, {}));
cursor.lineY.set('visible', false);


// Create axes
// https://www.amcharts.com/docs/v5/charts/xy-chart/axes/
var xRenderer = am5xy.AxisRendererX.new(root, { minGridDistance: 30 });
xRenderer.labels.template.setAll({
  rotation: -90,
  centerY: am5.p50,
  centerX: am5.p100,
  paddingRight: 15
});

xRenderer.grid.template.setAll({
  location: 1
})

var xAxis = chart.xAxes.push(am5xy.CategoryAxis.new(root, {
  maxDeviation: 0.3,
  categoryField: 'CustomerName',
  renderer: xRenderer,
  tooltip: am5.Tooltip.new(root, {})
}));

var yAxis = chart.yAxes.push(am5xy.ValueAxis.new(root, {
  maxDeviation: 0.3,
  renderer: am5xy.AxisRendererY.new(root, {
    strokeOpacity: 0.1
  })
}));


// Create series
// https://www.amcharts.com/docs/v5/charts/xy-chart/series/
var series = chart.series.push(am5xy.ColumnSeries.new(root, {
  name: 'Series 1',
  xAxis: xAxis,
  yAxis: yAxis,
  valueYField: 'QuotePrice',
  sequencedInterpolation: true,
  categoryXField: 'CustomerName',
  tooltip: am5.Tooltip.new(root, {
    labelText: '{valueY}'
  })
}));

series.columns.template.setAll({ cornerRadiusTL: 5, cornerRadiusTR: 5, strokeOpacity: 0 });
series.columns.template.adapters.add('fill', function(fill, target) {
  return chart.get('colors').getIndex(series.columns.indexOf(target));
});

series.columns.template.adapters.add('stroke', function(stroke, target) {
  return chart.get('colors').getIndex(series.columns.indexOf(target));
});

//get data

const element = document.querySelector(`[data-name='raw-data-custom2']`);
const text = element.textContent;
var data = JSON.parse(text);

xAxis.data.setAll(data);
series.data.setAll(data);


// Make stuff animate on load
// https://www.amcharts.com/docs/v5/concepts/animations/
series.appear(1000);
chart.appear(1000, 100);
    });
  });
});

">

[ConorWebb96]

Nice work @SufiSR and a fantastic write up!

[IchthysMaranatha]

Excellent! Informative and was very helpful in understanding how to get Budibase client side JS working.

Can be done using a single embed component. What is to be noted is that normal Budibase JS works server side and the onerror JS will work client side, so variables cannot be passed as variables, they will have to expressed fully as string literals. When doing so, they cannot contain quotes, so they must be replaced before expressing them. Have used the same code as @SufiSR , slightly modified for single embed as below, Have also attached the app export ClientJSBudi.tar.gz

const data = $("New Data Provider.Rows");
var transformedData = data.map(function(item) {
  return {
    CustomerName: item['Customer Name'],
    QuotePrice: item['Quote Price']
  };
});
data1=helpers.replace(JSON.stringify(transformedData),'"','"');
return `
  <img style="display : none;" src onerror="

var parentNode = document.querySelectorAll(\`[data-name='New Embed']\`);

function loadScript(url, callback) {
  var script = document.createElement('script');
  script.type = 'text/javascript';
  script.src = url;
  script.onload = callback;
 document.head.appendChild(script);
}

// Define the URLs of the amCharts script files
var amChartsLibURL = 'https://cdn.amcharts.com/lib/5/index.js';
var amChartsXYURL = 'https://cdn.amcharts.com/lib/5/xy.js';
var amChartsThemeURL = 'https://cdn.amcharts.com/lib/5/themes/Animated.js';

// Load the amCharts scripts in sequence
loadScript(amChartsLibURL, function() {
  loadScript(amChartsXYURL, function() {
    loadScript(amChartsThemeURL, function() {
      // All amCharts scripts are loaded
      // You can now initialize your amCharts chart

// Create a div element for the chart
var chartDiv = document.createElement('div');
chartDiv.id = 'chartdiv';
chartDiv.style.width = '100%';
chartDiv.style.height = '500px';
parentNode[0].appendChild(chartDiv);

// Create root element
// https://www.amcharts.com/docs/v5/getting-started/#Root_element
var root = am5.Root.new('chartdiv');

// Set themes
// https://www.amcharts.com/docs/v5/concepts/themes/
root.setThemes([
  am5themes_Animated.new(root)
]);

// Create chart
// https://www.amcharts.com/docs/v5/charts/xy-chart/
var chart = root.container.children.push(am5xy.XYChart.new(root, {
  panX: true,
  panY: true,
  wheelX: 'panX',
  wheelY: 'zoomX',
  pinchZoomX: true
}));

// Add cursor
// https://www.amcharts.com/docs/v5/charts/xy-chart/cursor/
var cursor = chart.set('cursor', am5xy.XYCursor.new(root, {}));
cursor.lineY.set('visible', false);

// Create axes
// https://www.amcharts.com/docs/v5/charts/xy-chart/axes/
var xRenderer = am5xy.AxisRendererX.new(root, { minGridDistance: 30 });
xRenderer.labels.template.setAll({
  rotation: -90,
  centerY: am5.p50,
  centerX: am5.p100,
  paddingRight: 15
});

xRenderer.grid.template.setAll({
  location: 1
})

var xAxis = chart.xAxes.push(am5xy.CategoryAxis.new(root, {
  maxDeviation: 0.3,
  categoryField: 'CustomerName',
  renderer: xRenderer,
  tooltip: am5.Tooltip.new(root, {})
}));

var yAxis = chart.yAxes.push(am5xy.ValueAxis.new(root, {
  maxDeviation: 0.3,
  renderer: am5xy.AxisRendererY.new(root, {
    strokeOpacity: 0.1
  })
}));


// Create series
// https://www.amcharts.com/docs/v5/charts/xy-chart/series/
var series = chart.series.push(am5xy.ColumnSeries.new(root, {
  name: 'Series 1',
  xAxis: xAxis,
  yAxis: yAxis,
  valueYField: 'QuotePrice',
  sequencedInterpolation: true,
  categoryXField: 'CustomerName',
  tooltip: am5.Tooltip.new(root, {
    labelText: '{valueY}'
  })
}));

series.columns.template.setAll({ cornerRadiusTL: 5, cornerRadiusTR: 5, strokeOpacity: 0 });
series.columns.template.adapters.add('fill', function(fill, target) {
  return chart.get('colors').getIndex(series.columns.indexOf(target));
});

series.columns.template.adapters.add('stroke', function(stroke, target) {
  return chart.get('colors').getIndex(series.columns.indexOf(target));
});

//get data

//const element = document.querySelector(\`[data-name='raw-data-custom2']\`);
//const text = element.textContent;
var data = JSON.parse(\`${data1}\`);

xAxis.data.setAll(data);
series.data.setAll(data);

// Make stuff animate on load
// https://www.amcharts.com/docs/v5/concepts/animations/
series.appear(1000);
chart.appear(1000, 100);
    });
  });
});

">`

[rishav-propertyguru]

Not sure why but it's not working anymore

[IchthysMaranatha]

Something blocking javascript on your browser? Does this work for you #6721 ?
Which budibase version? Public or self hosted?

[SufiSR]

That change is too bad.

One would have to modify https://github.com/Budibase/budibase/blob/e4eb09b203c0ef2d2dd11fd37f4cedff3219ab23/packages/backend-core/src/middleware/contentSecurityPolicy.ts#5

and ideally also the test that comes with it:

budibase/packages/backend-core/src/middleware/tests/contentSecurityPolicy.spec.ts

Line 35 in e4eb09b

`script-src 'self' 'unsafe-eval' https://*.budibase.net https://cdn.budi.live https://js.intercomcdn.com https://widget.intercom.io https://d2l5prqdbvm3op.cloudfront.net https://us-assets.i.posthog.com 'nonce-${mockNonce}'`

Either by pulling the project and then deploying that or modify the files directly.

[IchthysMaranatha]

There seems to be an environment variable DISABLE_CONTENT_SECURITY_POLICY processed here

budibase/packages/backend-core/src/environment.ts

Line 239 in e4eb09b

DISABLE_CONTENT_SECURITY_POLICY: process.env.DISABLE_CONTENT_SECURITY_POLICY,

which maybe could be used by setting it to 1 or TRUE in the env file. Haven't tried it myself.

[NicolasFabre81]

In fact we just need to store the .JS file locally, for example in a bb internal table attachment field, then it works !
PS : I do appreciate your username @IchthysMaranatha ;-)

[SufiSR]

Ooooh, Creative :)

[IchthysMaranatha]

I did check with setting DISABLE_CONTENT_SECURITY_POLICY=1 in the env file with self hosted docker. Stop and start container. It works.