SIsimonw·Jul 16, 2024
This is the code that does the work: https://github.com/rectanglehq/Shapeshift/blob/d954dab2a866c...
There are a few ways this could be made a less expensive to run:
1. Cache those embeddings somewhere. You're only embedding simple strings like "name" and "address" - no need to do that work more than once in an entire lifetime of running the tool.
2. As suggested here https://news.ycombinator.com/item?id=40973028 change the design of the tool so instead of doing the work it returns a reusable data structure mapping input keys to output keys, so you only have to run it once and can then use that generated data structure to apply the transformations on large amounts of data in the future.
3. Since so many of the keys are going to have predictable names ("name", "address" etc) you could even pre-calculate embeddings for the 1,000 most common
Read the rest
keys across all three embedding providers and ship those as part of the package.
Also: in https://github.com/rectanglehq/Shapeshift/blob/d954dab2a866c... you're using Promise.map() to run multiple embeddings through the OpenAI API at once, which risks tripping their rate-limit. You should be able to pass the text as an array in a single call instead, something like this:
const response = await this.openai!.embeddings.create({
model: this.embeddingModel,
input: texts,
encoding_format: "float",
});
return response.data.map(item => item.embedding);
https://platform.openai.com/docs/api-reference/embeddings/cr... says input can be a string OR an array - that's reflected in the TypeScript library here too: https://github.com/openai/openai-node/blob/5873a017f0f2040ef...