Feature Request: Server stream response for "prompt processing progress"

[BradHutchings]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

In the server tool, I would like the stream returned by the /completion endpoint to return "prompt processing progress" similar to what is displayed in the server log.

slot update_slots: id  0 | task 0 | prompt processing progress, n_past = 19200, n_tokens = 256, progress = 0.827836

Motivation

This would allow me to show evaluation progress to the user for long completion processes.

Possible Implementation

No response

[baonudesifeizhai]

[BradHutchings]

Dude! Thank you for taking this on and dealing with the reviewer feedback. This little feature will bring a smile to one of my client's faces!

[baonudesifeizhai]

Dude! Thank you for taking this on and dealing with the reviewer feedback. This little feature will bring a smile to one of my client's faces!

I see you've already opened a lot of issues — if they get merged, I’ll help you continue with the next steps.
By the way, I don’t need sponsorship and am searching for job open to relocate anywhere in USA

[BradHutchings]

I don't have paid work to offer right now. If you finish the work on this PR and get it merged, I'd be more than happy to acknowledge your work on my product web site, forked repo, etc. I'd also be happy to write a letter of recommendation if that would be helpful to you.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.