Davide
June 21, 2021, 6:59pm
1
Dear @lubos , I’m having an issue with API. I’m using n8n workflow software.
I’ve created a recursive script to query all.clients. starting from the index I try to call all the clients json. However in a group of 250 clients I get 30/40 errors. Some of the files randomly are not reachable. I get this log with an ECONNRESET error:
{ "name": "RequestError", "message": "Error: socket hang up", "cause": { "code": "ECONNRESET" }, "error": { "code": "ECONNRESET" }, "options": { "timeout": 3600000, "headers": { "accept": "application/json,text/*;q=0.99" }, "method": "GET", "uri": "http://192.168.1.77:8080/api/QU0gU3ZpbHVwcGkgSW1tb2JpbGlhcmk/05d0a9b6-c0bb-42ed-b0cb-958fa5d6d9ed.json", "gzip": true, "rejectUnauthorized": true, "resolveWithFullResponse": true, "json": true, "auth": { xxxx }, "simple": true, "transform2xxOnly": false } }
lubos
June 22, 2021, 12:39pm
2
This error is from n8n workflow software. Generally this shouldn’t be an error at all. If connection is closed, new should be created.
Davide
June 22, 2021, 9:02pm
3
Yes but it seems to be an issue of Manager since the n8n is able to get randomly 70/80% of the items. Every time the lines with an error change.
Is there a log somewhere of the API errors to understand where the problem could be.
lubos
June 22, 2021, 9:11pm
4
The only way to make sure is to remove n8n from the equation.
Try to refresh API endpoint in web-browser 250 times. If you can still see errors while using something else (e.g. web-browser) and not n8n, then this will determine what is at fault.
Davide
June 24, 2021, 5:20am
5
I did some testing. The issue that I see is that API are very resource angry, at least with Manager under Docker.
So after few queries (n8n uses many of them in parallel to speed up the process) my server reaches 100% of CPU use and I get errors.
Maybe having a unique list with all data (a combination of the index with all the files) could be a solution. @lubos ?
Patch
June 24, 2021, 5:38am
6
Sounds like a multitasking flow control issue.
You could just slow the communication a bit by adding wait commands
Davide
June 24, 2021, 5:51am
7
You are right and I will try it. But it is not normal that for a query of a single client manager eats more than 10% of a quad core CPU.
Patch
June 24, 2021, 6:13am
8
An API can interrogate a database much faster than a few busy users. Especially if the other software was not waiting for a response.
Davide
June 24, 2021, 7:08am
9
Just to give you an example. On the other side I have the e-invoice portal to sync which I query with an API too. On the other side I have an answer in less than one second with all the invoices.
MarkLL
June 24, 2021, 11:44am
10
So I have been observing the changes to the system and as one of the major Pros (but to me is a con) is the fact that the software is evolving very rapidly. From experience I have found that it’s always best to create a “test harness” of some kind so you can catch when breaking changes have been made.
With this in mind, I have a utility that “walks” the API tree for a given file to check how many entries are there etc.
$>art -v api:walk Northwind
Walking the API Tree of business Northwind
Using Host Name http://localhost:8123
Wait... Commencing walk. . .
100 successful calls so far
200 successful calls so far
...
8600 successful calls so far
Here are the API Links available for Northwind
+-------------------------------------------------+-------------------------------------------------------------------+-------+
| Name | Link | Items |
+-------------------------------------------------+-------------------------------------------------------------------+-------+
| .. | /api/index.json | 9 |
| AgedPayables | /api/Tm9ydGh3aW5k/6abe86ec-a6c9-46a8-a9c9-7104056a2730/index.json | 0 |
| AgedReceivables | /api/Tm9ydGh3aW5k/39f00628-27a7-4924-9030-5bc655ee234f/index.json | 0 |
| BalanceSheet | /api/Tm9ydGh3aW5k/7b4f463a-470d-44c4-9e75-fafc630b5851/index.json | 0 |
| BankAccount | /api/Tm9ydGh3aW5k/1408c33b-6284-4f50-9e31-48cbea21f3cf/index.json | 1 |
| BusinessLogo | /api/Tm9ydGh3aW5k/096d0af9-df72-425d-aae8-d59c0497f119/index.json | 0 |
| CashAccount | /api/Tm9ydGh3aW5k/25064eec-02ab-46f3-b71f-f8e78ad4ca45/index.json | 1 |
| CreditNote | /api/Tm9ydGh3aW5k/245e5943-0092-409d-96ae-e2ee10eac75b/index.json | 44 |
| Customer | /api/Tm9ydGh3aW5k/ec37c11e-2b67-49c6-8a58-6eccb7dd75ee/index.json | 91 |
| CustomerPortal | /api/Tm9ydGh3aW5k/18048748-7c70-49e6-bed4-b9d310736956/index.json | 2 |
| CustomField | /api/Tm9ydGh3aW5k/dcb382dc-a4e0-4354-a845-b7d647f610f7/index.json | 3 |
| DateAndNumberFormat | /api/Tm9ydGh3aW5k/a56e89d1-7bee-4509-8b84-c9ebc3808b0c/index.json | 1 |
| GeneralLedgerSummary | /api/Tm9ydGh3aW5k/6c1d3132-7978-45c8-a6e2-2387c7de46b0/index.json | 0 |
| GeneralLedgerTransactions | /api/Tm9ydGh3aW5k/a3283b79-76be-44b6-9639-fa22d9b63246/index.json | 1 |
| InventoryItem | /api/Tm9ydGh3aW5k/0dbdbf8a-d80c-48e6-b453-bb7862445b7c/index.json | 0 |
| InventoryLocation | /api/Tm9ydGh3aW5k/fae8151d-252e-45e3-b1f4-e048075b8983/index.json | 2 |
| Payment | /api/Tm9ydGh3aW5k/79f99d26-e43a-4ecb-a9c9-0774601a9b2e/index.json | 3831 |
| ProfitAndLossStatement | /api/Tm9ydGh3aW5k/165c0392-9aad-4067-b855-a2393ead5df4/index.json | 0 |
| ProfitAndLossStatementAccount | /api/Tm9ydGh3aW5k/26b9e4a5-ce10-4f30-94c7-23a1ca4428f9/index.json | 0 |
| ProfitAndLossStatementAccountInventoryPurchases | /api/Tm9ydGh3aW5k/aa80b662-3642-4c08-b328-2fccf132ceb1/index.json | 1 |
| ProfitAndLossStatementAccountInventorySales | /api/Tm9ydGh3aW5k/ea44f579-9548-4954-baf0-48538aceff1e/index.json | 1 |
| ProfitAndLossStatementGroup | /api/Tm9ydGh3aW5k/5770616c-0e01-46ca-a172-f7042275da6c/index.json | 5 |
| PurchaseInvoice | /api/Tm9ydGh3aW5k/58b9eb90-f6b8-4abc-8ea1-12fd77b8336e/index.json | 1449 |
| Receipt | /api/Tm9ydGh3aW5k/7662b887-c8d8-486e-98fd-f9dbcd41c6dc/index.json | 1190 |
| SalesInvoice | /api/Tm9ydGh3aW5k/ad12b60b-23bf-4421-94df-8be79cef533e/index.json | 1999 |
| Subtotal | /api/Tm9ydGh3aW5k/9601ce49-6058-4dac-9405-82f35005ea90/index.json | 3 |
| Summary | /api/Tm9ydGh3aW5k/2631d044-861d-4710-a871-d7a11461b4ba/index.json | 0 |
| Supplier | /api/Tm9ydGh3aW5k/6d2dc48d-2053-4e45-8330-285ebd431242/index.json | 29 |
| Tabs | /api/Tm9ydGh3aW5k/ac789d1f-034f-4964-a8b5-ebfffc3511f2/index.json | 1 |
| TrialBalance | /api/Tm9ydGh3aW5k/e5dc98ef-4662-4a68-8a9d-b3e2d12b55d6/index.json | 0 |
+-------------------------------------------------+-------------------------------------------------------------------+-------+
Done: Success calls = 8686
Failed Calls : 0
With Data : 20
Total Time spent was 21 minutes
So I think the API is responding OK as this works out to about 6.8 calls completed per second!
It’s also helped me notice the changes made recently to the naming convention for fields in addition to identifying areas where the program fails to update the API reading in old files. (If you check the Northwind test database, you will notice that none of the Inventory items are listed in the API (/0dbdbf8a-d80c-48e6-b453-bb7862445b7c/index.json has no entries) and if you view the page in the browser you will notice that the UUIDs have 0 next to them.
Hopefully it will remain stable and not have too many breaking changes to come.
Davide
June 24, 2021, 6:01pm
11
I will investigate more. Maybe it is something due to Dockers’ virtual machine.
MarkLL
June 25, 2021, 6:20am
12
Just as an FYI, I run ManagerServer in a WSL2 Ubuntu 18.04 VM on my dev machine. I also run Nginx (on my Windows machine, not the WSL instance) as the front server with a very basic config. Very easy to setup and run. You can also see how I override the default favicon.ico as well
# Virtual Hosts - dev.miotest.local - https
# Config for secure connection to Manager.io ManagerServer instance
#
# https://dev.miotest.local/
server {
listen 80;
listen 443 ssl http2;
server_name dev.miotest.local;
access_log logs/$host.access.log;
ssl_certificate "d:/wamp/certs/dev.miotest.local.chained.crt";
ssl_certificate_key "d:/wamp/certs/dev-pc.key";
location /favicon.ico {
root D:/Dev/MarkLL/wip/ManagerApi/public_html;
}
location / {
proxy_pass http://localhost:8123;
}
}
Davide
June 25, 2021, 4:50pm
13
Thanks. As said changing favicon is much easier…
.