undici

If anyone is curious how they are measuring these are the benchmarks: https://github.com/nodejs/undici/blob/main/benchmarks/benchm... I did some testing on an M3 Max Macbook Pro a couple of weeks ago. I compared the local server benchmark they have against a benchmark over the network. Undici appeared to perform best for local purposes, but Axios had better performance over the network. I am not sure why that was exactly, but I have been using Undici with great success for the last year and a half regardless. It is certainly production ready, but often requires some thought about your use case if you're trying to squeeze out every drop of performance, as is usual.

Not only do they not respect robots.txt, but they publish an entire page[1] in their docs dedicated to circumventing scraping countermeasures. I pointed their scraper at a url on my server to test it's behavior. It made four separate requests to the same page, three with the UA "udici"[2], and one with the UA "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36" and a different IP. The first three requests were all made within the span of 1 second, and the fourth 27 seconds later. I emailed their published support address asking for an IP range and UA. They gave me the entire IP range of google cloud, and ignored the UA question. This goes well beyond the "it's up to our users to implement responsible scraping practices" implication from the developer's other comment[3]. Instead, their ser

I'm surprised it didn't mention Undici, which is included in Node.js and has massive usage. (Albeit most of it in the form of the fetch() wrapper.) https://github.com/nodejs/undici

Yeah: interestingly, they had a test for the biggest category of frame, but not for the two other categories: https://github.com/nodejs/undici/blob/main/test/websocket/se... The test I contributed is very specific to the frame fix I made, but I should probably go back and contribute more tests in send.js that test other lengths too.

> ...enough insights of how much better/faster Deno is We moved our Deno project to Node because of lack of lower-level APIs on Deno's Conn interfaces [0][1], but otherwise for our use-case (lots of tiny HTTPS connections) Deno absolutely blew Node out-of-the-water. Even at p50 (100tps) Deno (v1.18) was 10x faster than Node (v17.x) [2] RAM wise, I found Deno (v1.18+) use 10M or so higher for the same code-base. DevEx wise, Node is miles away from Deno, but it might get there in-time, or it might not [3]. I'd imagine, what may also work in Deno's favour in terms of IO-bound workloads, is the underlying tokio layer (vs libuv in Node). API wise, Deno (comparatively) feels quite raw. [0] https://github.com/denoland/deno/issues/13636 [1] https://github.com/denoland/deno/issues/9109 [2] https://github.com/nodejs/undici/issues/1203#issuecomment-

Thanks. Filed: https://github.com/nodejs/undici/issues/1203

That URL gives me a 404 – should it be https://github.com/nodejs/undici ?

I love fetch, except for the way that it combines headers. If more than one of the same header is set (such as multiple set-cookie headers), it combines them all into a single comma-separated string. I know this is allowed in the HTTP spec, and it's probably even a sensible default. But the fetch spec doesn't allow any access to the raw headers, so there's no straightforward way to get the original uncombined headers back. Set-Cookie headers, in particular, often contain commas in their expiration dates, so just splitting around `, ` will lead to problems. I took a look through the source of this new fetch API, and it seems to have inherited that wart: https://github.com/nodejs/undici/blob/2dd3437e20c5a3cc466226... I've argued with the authors of the fetch spec about this before, and ultimately settled on using https://www.npmjs.com/pack

Those wasm blobs are Node's own llhttp https://github.com/nodejs/llhttp in wasm to speed up HTTP parsing. The project itself added as a dependency is also by Node https://github.com/nodejs/undici . The question is totally legitimate but please assume core doesn't make "load random binary" level kind of goofs :)

Is there any chance to break spec and allow manual redirect handling? the fetch API makes a lot of sense in a browser, but imo this is a pretty crucial feature that undici's implementation lacks. [0] Deno decided to break spec [1][2] so the following code works fine: fetch('https://httpbin.org/status/302', {redirect: 'manual'}) .then(res => console.log(res.status, res.headers)) In undici this will succeed but with res.status 0 and no headers, as per spec. You aren't allowed to see the content of a redirect, like in a browser. [0] https://github.com/nodejs/undici/issues/1072 [1] https://github.com/denoland/deno/pull/8353 [2] https://github.com/denoland/deno/issues/4389

The commit adds undici (another Node.js project at https://github.com/nodejs/undici ) as a dependency and exposes its `fetch` - the code changes you see are probably just adding a flag :) > I just wonder how come features like this that kind of seems obvious to include in the ecosystem takes quite some time to land? I answered that below (check it out) but note how expensive adding a bad API is vs. asking people for one more `npm install` :) There is more discussion in https://github.com/nodejs/node/issues/19393 and in https://docs.google.com/document/d/1tn_-0S_FG_sla81wFohi8Sc8... a discussion from 2018 we had on it