I am using openAI Assistant, I get the streamingUpdate using
streamingUpdate = assistantClient.CreateRunStreaming(thread, assistant);
I could iterate through streamingUpdate and send the response to the client using SignalR api. however I would like to know how to count the #of tokens consumed in the streaming mode.
I am using openAI Assistant, I get the streamingUpdate using
streamingUpdate = assistantClient.CreateRunStreaming(thread, assistant);
I could iterate through streamingUpdate and send the response to the client using SignalR api. however I would like to know how to count the #of tokens consumed in the streaming mode.
Share Improve this question edited Jan 8 at 19:39 General Grievance 5,04338 gold badges37 silver badges56 bronze badges asked Nov 18, 2024 at 10:14 Sankaranarayanan RajkumarSankaranarayanan Rajkumar 511 silver badge7 bronze badges1 Answer
Reset to default 1I found the way to get the token count
CompletionTokens = assistantClient.GetRuns(threadID).Select(f => f.Usage.CompletionTokens).SingleOrDefault();
PromptTokens = assistantClient.GetRuns(threadID).Select(f => f.Usage.PromptTokens).SingleOrDefault();
TotalTokens = assistantClient.GetRuns(threadID).Select(f => f.Usage.TotalTokens).SingleOrDefault();