Volume management

Feature management has settings dedicated to configuration for entities with huge volumes (typically, EDR, WalletOperation, RatedTransaction, InvoiceLines, …).

Based on these settings, API and Portal will adapt their behaviour.

Huge entities

This section allows to identify entities with huge volumes and Portal lists to adapt.

Configuration

In General settings, on feature management page, scroll down to section “Volume management”.

Entities identified as “huge”
Lists in Portal that are impacted
(more details about “Finding Portal list IDs” below)
Specify mandatory filters (force the user to fill this filters before fetching data)
(more details about “Finding Portal filter IDs” below)
Add a new entity to the list of entities
Remove an entity from the list

Return to the top of the page and save your work

Finding Portal list IDs

As you can see in the screenshot above, the impacted lists are listed by their ID.
This configuration is expected to be done by the integrator or someone from a computer department, so we decided to put our effort elsewhere. 😉

That being said, here how to find the information we need.

List IDs have been added directly to the Portal HTML code so it can easily be accessed by the user:

Go to the page where the list is
(remember that you can open the Portal in several tabs)
Open the dev tools (for Chrome or Firefox on Windows, hit F12 or CTRL+SHIFT+I)
Go to the HTML tab (“Elements” in Chrome, “Inspector” in Firefox)
Search (CTRL+F) for listid (added as property on some <div> tags)
Make sure you’re on the correct list (using the inspector’s highlighting)
Copy the value of property listid
Paste it in the list for the corresponding entity in “Volume management”
Save

You did it !

Notes

To add several lists for the same entity, separate IDs using a comma ,
It can happen that the same ID is on several tags.
Don’t worry about it, as long a you can see the correct list highlighted.

Dynamic nested lists

Some lists are dynamic, typically the lists of rated items nested under invoice lines on the invoice details page:

These list are dynamically created on the page, as you have 1 “sub list” for each invoice line.
So, we can’t add all the potential dynamically computed values of list_id to the list of huge lists.

For this specific case, we have added a special listid: B2B-customer-care/invoice-lines_NESTED_RTS_LIST
It will tell the Portal to consider all rated items sub-lists as “huge”.

Finding Portal filters IDs

Finding filter ID is similar to finding a list ID, so you can refer to the previous section for more details.

The only difference is that you’ll have to look for filterid (instead of listid), and you will have as many filterid as you have displayed filters (optional filter will not show up until they are selected).

Impact on API

APIs, specifically generic API and API based on filters will behave differently if the entity they are working on is in the “huge entities” list.

Generic API will return a total count for the result set unless specifically request by parameter forceCount:true
(as count requires to scan all table to count all results)
JSON
```
POST /v2/generic/all/WalletOperation
{
    "limit": 10,
    "offset": 0,
    "forceCount": true
}
>>>
{
  "total": 328041,
  "limit": 10,
  "offset": 0,
  "data": [
  ...
  ]
}
```
Please, note that a dedicated generic count API has been provided.
Other APIs will force an async operation (using a job) or refuse to process the request.
(more info on each feature’s documentation)

Impact on Portal

Lists in Portal will undergo the following behaviour changes:

No sort allowed (as sorting is a very costly operation on huge volumes)
No default sort applied
Total record count is not displayed (as count is also a costly operation)
No data displayed on page load
No automatic query on filter update
It is required to click on a “Filter” button to send query
“Contains” filters are replaced by filters with operators (“exact match”, “starts with”, etc.)
… (other adjustments may be added in the future if need be)