empirical-philosophy

One way that I've been framing this in my head (and in an application I'm building) is that gpt-3 will be useful for analytic tasks but gpt-4 will be required for synthetic tasks. I'm using "analytic" and "synthetic" in the same way as in this writeup https://github.com/williamcotton/empirical-philosophy/blob/m...

I don’t really understand your criticism but I’d be happy to continue a dialog to find out why you mean! There’s probably a little too much going on with that project, including generating datasets for fine-tuning, which is the reason for comparing with a known answer. It is very similar to the approach used by the Toolformer team. But by teaching an agent to use a tool like Wikipedia or Duck Duck Go search it dramatically reduces factual errors, especially those related to exact numbers. Here’s a more general overview of the approach: From Prompt Alchemy to Prompt Engineering: An Introduction to Analytic Augmentation https://github.com/williamcotton/empirical-philosophy/blob/m...

Hey Simon! I've been digging your writings on LLMs lately. I've been having some decent luck with some of the approaches that I've discussed in the following articles and projects: From Prompt Alchemy to Prompt Engineering: An Introduction to Analytic Augmentation: https://github.com/williamcotton/empirical-philosophy/blob/m... Writing Web Applications with LLMs: https://www.williamcotton.com/articles/writing-web-applicati... https://github.com/williamcotton/transynthetical-engine I'd love to hear your thoughts on the matter! One of the techniques that I've found for reliably returning JSON is... ask for multiple responses and then use one of the responses that successfully parses!

You’re not actually doing any research. Here is my research: https://github.com/williamcotton/empirical-philosophy/blob/m... It is clear that analytic augmentations will result in more factual information. Your claims are unfounded and untested.

You call it lying because you don’t understand how it works. https://github.com/williamcotton/empirical-philosophy/blob/m... https://williamcotton.com/articles/chatgpt-and-the-analytic-...

https://github.com/williamcotton/empirical-philosophy/blob/m... You are very wrong about how these things work.

Take a look at https://github.com/williamcotton/empirical-philosophy/blob/m... https://langchain.readthedocs.io/en/latest/ They can be taught!

It is both a reliable synthesizer of fiction and a translator of fact, and empirically so: https://github.com/williamcotton/empirical-philosophy/blob/m...

I dunno what to say, but I’m also someone who very rarely gets factually incorrect data aside from numbers, dates and quotes. I make a point to ask for a completion that I know won’t depend on an exact factoid. If I do need an exact factoid, I have a tool I made myself based on this approach: https://github.com/williamcotton/empirical-philosophy/blob/m...

It’s more like this: subprompts don’t ever inject the full context from a remote query back into the primary prompt. The completions of subprompts are (via few-shot or a fine-tuned model) structured, eg, JSON, which is then parsed. The main prompt is orchestrating the subprompts and never needs to even process the results if there’s a Python or JS interpreter involved. Here’s the kind of approach I’ve been using: https://github.com/williamcotton/empirical-philosophy/blob/m... The initial call to the LLM will return a completion that includes JavaScript. There is no third-party data at this point. The JavaScript includes further calls to the LLM that returns JSON, but at this point no further calls are made to the LLM. This means that responses from remote queries are never sent to an LLM. The text presented to the user could be some instr

It can be made more reliable. Here’s how: https://github.com/williamcotton/empirical-philosophy/blob/m...

I would imagine that those are not actual academic papers and that the ChatGPT consumer web application is not the right tool. Try using the OpenAI APIs directly, but through few-shot examples train it to return some interpreted language like Python or JS, which has the ability to execute further completions, but with the added context of a search from a web search or better yet the search results from an academic journal search engine. Here’s the general approach: https://github.com/williamcotton/empirical-philosophy/blob/m...

Here is an objective and critical assessment of large language models and how to empirically improve the results of questions involving calculations like the lengths of strings: https://github.com/williamcotton/empirical-philosophy/blob/m...

It's not that hard: https://github.com/williamcotton/empirical-philosophy/blob/m...