Skip to content

[PROFILE] added timers for serialization/deserialization#3889

Closed
ExtReMLapin wants to merge 1 commit into
ArcadeData:mainfrom
ExtReMLapin:serialization_timers
Closed

[PROFILE] added timers for serialization/deserialization#3889
ExtReMLapin wants to merge 1 commit into
ArcadeData:mainfrom
ExtReMLapin:serialization_timers

Conversation

@ExtReMLapin

Copy link
Copy Markdown
Contributor

Just a demo pr for

gemma 27b wrote all of it, ref #3864 (comment)

Protocol Overhead:
- Deserialization: 0,042 ms
- Engine Execution: 7,789 ms
- Serialization: 78,219 ms
- Total Overhead: 78,262 ms
OpenCypher Query Profile
========================

Execution Time: 3,302 ms
Rows Returned: 1906

Execution Plan (Cost-Based Optimizer):
+ NodeByLabelScan(n:NER) [cost=1906.00, rows=1906]

Estimated Cost: 1906,00
Estimated Rows: 1906

@codacy-production

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 1 medium

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category Results
BestPractice 1 medium

View in Codacy

🟢 Metrics 966 complexity

Metric Results
Complexity 966

View in Codacy

TIP This summary will be updated as you push new changes. Give us feedback

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces performance profiling to measure protocol overhead—specifically deserialization, engine execution, and serialization times—across the Bolt, Gremlin, gRPC, and HTTP interfaces. While the addition of these metrics is valuable for debugging, several issues were identified regarding the accuracy and implementation of the timers. Specifically, some timers measure negligible operations or are misplaced relative to the actual work being performed. Additionally, there is unused code in the Gremlin implementation and duplicated string formatting logic in the HTTP handler that should be refactored for better maintainability.

Comment on lines +514 to +516
long deserializationStart = System.nanoTime();
// Note: BoltMessage.parse already did the main deserialization, but we can consider the map creation as part of it
long deserializationTime = System.nanoTime() - deserializationStart;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This timer is misleading as it measures almost no operation (only the time between two consecutive System.nanoTime() calls). The actual deserialization of the BOLT message occurs earlier in BoltMessage.parse (line 180). Furthermore, the "map creation" mentioned in the comment happened at line 508, before this timer started. To provide accurate metrics, the start time should be captured before the message is read from the input stream in the main loop.

Comment on lines +150 to 159
long serializationTime = System.nanoTime() - serializationStart;

if (profileExecution && executionPlan != null) {
// We can't easily modify the executionPlan object because it's an anonymous inner class
// and prettyPrint is called later. But we could potentially store the times
// and use them in the prettyPrint implementation (which we already did).
// To include serialization time, we'd need to pass it into the anonymous class.
}

return result;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The serializationTime variable is calculated but never used. Additionally, the timer at line 123 only measures the instantiation of IteratorResultSet, not the actual serialization of results, which happens during iteration. The empty if block and the unused variable should be removed to clean up the code.

Suggested change
long serializationTime = System.nanoTime() - serializationStart;
if (profileExecution && executionPlan != null) {
// We can't easily modify the executionPlan object because it's an anonymous inner class
// and prettyPrint is called later. But we could potentially store the times
// and use them in the prettyPrint implementation (which we already did).
// To include serialization time, we'd need to pass it into the anonymous class.
}
return result;
return result;

}
}

long serializationTime = 0;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Setting serializationTime to a hardcoded 0 is inconsistent with the if (returnRows) branch. Even when rows are not returned, the ResultSet is still drained (lines 356-367), which involves processing overhead. Reporting exactly 0.000 ms in the profile output is misleading; the time should be measured around the loop that consumes the results.

Comment on lines +139 to +142
String overhead = String.format(
"\nProtocol Overhead:\n- Deserialization: %.3f ms\n- Engine Execution: %.3f ms\n- Serialization: %.3f ms\n- Total Overhead: %.3f ms\n",
deserializationTime / 1_000_000.0, engineTime / 1_000_000.0, serializationTime / 1_000_000.0, (deserializationTime + serializationTime) / 1_000_000.0);
response.put("explain", overhead + explainText);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The logic for formatting the Protocol Overhead string is duplicated here and at lines 157-159. This should be refactored into a private helper method to improve maintainability and ensure consistency in the output format across different execution paths.

@lvca

lvca commented Apr 17, 2026

Copy link
Copy Markdown
Member

This is a great idea. I'll sue this concept to draft a commit quickly.

@lvca lvca closed this Apr 17, 2026
@lvca lvca added this to the 26.4.1 milestone Apr 17, 2026
@lvca lvca self-assigned this Apr 17, 2026
tae898 pushed a commit to humemai/arcadedb-embedded-python that referenced this pull request Jun 28, 2026
…io [skip ci]

Bumps [marked](https://github.com/markedjs/marked) from 15.0.12 to 17.0.4.
Release notes

*Sourced from [marked's releases](https://github.com/markedjs/marked/releases).*

> v17.0.4
> -------
>
> [17.0.4](markedjs/marked@v17.0.3...v17.0.4) (2026-03-04)
> -----------------------------------------------------------------------------------
>
> ### Bug Fixes
>
> * prevent ReDoS in inline link regex title group ([ArcadeData#3902](https://redirect.github.com/markedjs/marked/issues/3902)) ([46fb9b8](markedjs/marked@46fb9b8))
>
> v17.0.3
> -------
>
> [17.0.3](markedjs/marked@v17.0.2...v17.0.3) (2026-02-17)
> -----------------------------------------------------------------------------------
>
> ### Bug Fixes
>
> * escape image alt text ([ArcadeData#3896](https://redirect.github.com/markedjs/marked/issues/3896)) ([909fe44](markedjs/marked@909fe44))
>
> v17.0.2
> -------
>
> [17.0.2](markedjs/marked@v17.0.1...v17.0.2) (2026-02-11)
> -----------------------------------------------------------------------------------
>
> ### Bug Fixes
>
> * fix blockquote after list ([ArcadeData#3888](https://redirect.github.com/markedjs/marked/issues/3888)) ([2a475a1](markedjs/marked@2a475a1))
> * fix empty list item ([ArcadeData#3890](https://redirect.github.com/markedjs/marked/issues/3890)) ([3fc6a44](markedjs/marked@3fc6a44))
> * fix list item wrong indent ([ArcadeData#3889](https://redirect.github.com/markedjs/marked/issues/3889)) ([e031175](markedjs/marked@e031175))
> * fix list with tabs ([ArcadeData#3891](https://redirect.github.com/markedjs/marked/issues/3891)) ([9fc4f8e](markedjs/marked@9fc4f8e))
> * fix strikethrough flanking rules ([ArcadeData#3882](https://redirect.github.com/markedjs/marked/issues/3882)) ([1a5b124](markedjs/marked@1a5b124))
>
> v17.0.1
> -------
>
> [17.0.1](markedjs/marked@v17.0.0...v17.0.1) (2025-11-20)
> -----------------------------------------------------------------------------------
>
> ### Bug Fixes
>
> * fix block elements in task item ([ArcadeData#3828](https://redirect.github.com/markedjs/marked/issues/3828)) ([921ee22](markedjs/marked@921ee22))
>
> v17.0.0
> -------
>
> [17.0.0](markedjs/marked@v16.4.2...v17.0.0) (2025-11-07)
> ===================================================================================
>
> ### Bug Fixes
>
> * only create tokens inside tokenizers ([ArcadeData#3755](https://redirect.github.com/markedjs/marked/issues/3755)) ([7b19231](markedjs/marked@7b19231))
>
> ### BREAKING CHANGES
>
> * Change how consecutive text tokens work in lists
> * Simplify listItem renderer
> * Checkbox token is added in list tokenizer

... (truncated)


Commits

* [`22f0c55`](markedjs/marked@22f0c55) chore(release): 17.0.4 [skip ci]
* [`46fb9b8`](markedjs/marked@46fb9b8) fix: prevent ReDoS in inline link regex title group ([ArcadeData#3902](https://redirect.github.com/markedjs/marked/issues/3902))
* [`5b6faee`](markedjs/marked@5b6faee) chore(deps-dev): Bump eslint from 10.0.1 to 10.0.2 ([ArcadeData#3904](https://redirect.github.com/markedjs/marked/issues/3904))
* [`bcdaf6a`](markedjs/marked@bcdaf6a) chore(deps-dev): Bump `@​semantic-release/npm` from 13.1.4 to 13.1.5 ([ArcadeData#3905](https://redirect.github.com/markedjs/marked/issues/3905))
* [`baa78a5`](markedjs/marked@baa78a5) docs: Add marked-abc to known extensions list ([ArcadeData#3903](https://redirect.github.com/markedjs/marked/issues/3903))
* [`1aed9ac`](markedjs/marked@1aed9ac) chore(deps-dev): Bump eslint from 10.0.0 to 10.0.1 ([ArcadeData#3901](https://redirect.github.com/markedjs/marked/issues/3901))
* [`8045055`](markedjs/marked@8045055) chore: rename escape helper function ([ArcadeData#3900](https://redirect.github.com/markedjs/marked/issues/3900))
* [`bced615`](markedjs/marked@bced615) chore(release): 17.0.3 [skip ci]
* [`909fe44`](markedjs/marked@909fe44) fix: escape image alt text ([ArcadeData#3896](https://redirect.github.com/markedjs/marked/issues/3896))
* [`eb8ba2b`](markedjs/marked@eb8ba2b) chore(deps-dev): Bump `@​semantic-release/github` from 12.0.5 to 12.0.6 ([ArcadeData#3897](https://redirect.github.com/markedjs/marked/issues/3897))
* Additional commits viewable in [compare view](markedjs/marked@v15.0.12...v17.0.4)

Maintainer changes

This version was pushed to npm by [GitHub Actions](<https://www.npmjs.com/~GitHub> Actions), a new releaser for marked since your current version.

  
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility\_score?dependency-name=marked&package-manager=npm\_and\_yarn&previous-version=15.0.12&new-version=17.0.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
  
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot show  ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants