Premium support for our pure JavaScript UI components


Post by H9FFDC »

Hi,

We have noticed some performance issues when using the Grid in a Salesforce LWC with high volumes of data. Specifically, loading in 5000 records into the grid is taking roughly 20 seconds.

Our investigations have shown that this issue will occur when the data being set on the Grid is a proxied array from the Lightning Locker Service. This is due to a limitation with how proxied arrays are implemented in the Salesforce Lightning Locker Service. We have identified that when you access a single element from a proxied array, then the whole array is iterated over. This means that if you implement a standard

for (let i = 0; i < count; i++)

loop over an array, then the total number of operations is count * count. For an array with 5000 elements, this results in 25,000,000 operations.

It is therefore much more efficient to iterate over arrays using other methods such as forEach() or map(). I have attached a code snippet which demonstrates the issue with arrays in the Lightning Locker Service. We have reported this to Salesforce, but we have doubts as to whether they will be able to change their implementation of the Locker Service to mitigate this issue.

With regards to the grid - we can see that the loadData() function does a standard array loop, which has a heavy performance impact if the data is a Locker Service proxied array. If this were to be changed to be using the Array.forEach() function, then the performance would be vastly improved.

Thanks.

Attachments
Screenshot 2020-11-18 at 11.58.33.png
Screenshot 2020-11-18 at 11.58.33.png (71.22 KiB) Viewed 2070 times

Post by Maxim Gorkovsky »

Hello.
Thank you for report, I opened a ticket to track this issue: https://github.com/bryntum/support/issues/1899

After an internal discussion we decided that is it unreasonable for us to refactor codebase in general to work around broken for loop in LWC applications. I can see few solutions though:

  1. Clone data, provided by Salesforce, to a simple array of objects without side effects, using effective forEach, and then provide that data to the grid.
const data = [];
this.array.forEach(x => data.push(cloneRecord(x)))

new Grid({
  data : data
})
  1. Override troubled store methods which use for loop and replace it with forEach. There could be plenty of places, we use for loop a lot because often it is more suitable and effective than forEach. Problems could be not only during load, but also serialize, sort, filter, insert etc.

  2. Hire us to conduct this investigation and create special bundle for LWC which relies on forEach loops.

Frankly speaking, first option (clone data) appears most viable: using plain JS objects/arrays you make sure there are no hidden side effects. Although you would have to manually move changes from grid store to Salesforce data.

PS:
Can you please check what exactly is the bottleneck? Accessing array length or array item? Does array work faster if you store length variable too:

for (let i = 0, l = this.array.length; i++) {
  this.doSomethingWithElement(this.array[i]);
}

In case problem is reading length only and it fixes the performance issue, we'd need to have another discussion internally.


Post by H9FFDC »

Hi,

Thanks for taking the time to discuss this and for your detailed response.

We are planning on doing something similar to your first suggestion to ensure that we pass a non proxied array to the grid. We appreciate that this is not an issue in your code, however we wanted to make you aware of the issue.

With regards to your second question, the bottleneck is in accessing an array item. Internally, it appears that the Locker Service iterates over the whole array every time you access an array item.

Thanks.


Post by gbrdhvndi »

Thank you for your response, Maxim.

I'll try the override for loadData to see if it makes a significant difference for us.

As for your second proposal, I understand you already ship an LWC-specific build in the bundle (which we adopted since v4.0.0 came out) and assumed that it would not be too much trouble to make some minor tweaks in your code base like this for→forEach replacement to cater for LWC Proxy restrictions without compromising the main code base too much.

Cheers

Aleksei

Aleksei


Post by Maxim Gorkovsky »

Aleksei,
in the related issue I posted a performance measurement of iterating with forEach and forloops. Performance loss of forEach suggests we cannot make this change to the main code base, which means it should be an LWC override. We will look into that. But as we have a lot in the pipeline now, this could take a while.

I'll try the override for loadData to see if it makes a significant difference for us.

Assuming problem is only with loading the data set, where you iterate array provided by the LWC API, this approach sounds reasonable. Please let us know of the result, or if you need assistance overriding the Store.

Also, could you please link us to this issue reported to Salesforce? In case we will provide an override, it would be good to know when we can remove it.
Thanks.


Post by gbrdhvndi »

Maxim,

Salesforce responded that they have been able to replicate the performance issue on their end but due to the fundamental design of their data proxying feature they are not going to do anything to fix it, nor they were able to suggest any changes in the third party code (your Grid) as they do now own it, which is fair enough.

I did some preliminary testing and was able to load 4800 records into the grid in just over 1 second by going through the store: grid.store.add(records). The performance is good even if the records array is a Proxy object. Although my example was an artificial one and is not representative of an actual product.

I will work with the team which reported the issue and let you know if we have resolved it on our end.

Thanks,
Aleksei

Aleksei


Post by Maxim Gorkovsky »

Aleksei,

Thank you for the update. We now realize there is no way this issue will be fixed without our intervention.

May I ask why you chose not to iterate over proxy data object and cloning records to the native Array? Are there downsides?


Post by gbrdhvndi »

Maxim,

In my artificial test example I tried both: use the Proxy as is as well as convert it into a plain array first, just to see the difference.

Proxied arrays are read-only so if we want to use Grid's cell editing capabilities then conversion is a must.

As I mentioned above, I will work closely with the team to understand where the differences are between my demo and their product.

One notable difference I can see right now from what has been reported by my colleague in this thread is that they assign data to the Grid at the time of construction where as my demo component creates a grid with no data and then goes through the store to add records.

The store does not iterate using a for loop but instead does something like this.records = [...records] if I'm not mistaken.

Aleksei


Post Reply