-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
bugSomething isn't workingSomething isn't workingnodePeer-to-peer node features and optimizations.Peer-to-peer node features and optimizations.torchPyTorch neural network workflow and optimizations.PyTorch neural network workflow and optimizations.
Description
Two issues have arose when sending generate requests via API. The model output can sometimes be cut short, perhaps just an artifact of the LLMs being used, perhaps not. Likewise sometimes the request gets stuck on the worker (ie. Validator receives request, sends .generate to worker, who does not return anything and we get a 500 internal server error response.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingnodePeer-to-peer node features and optimizations.Peer-to-peer node features and optimizations.torchPyTorch neural network workflow and optimizations.PyTorch neural network workflow and optimizations.