Jump to content

This is a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Talk : CDN

From Wikitech

Should we include software version numbers?

This seems easy to forget to update, causing out-of-date information. Would it be simpler to just say the software used without the version numbers? BCornwall ( talk ) 21:14, 6 February 2023 (UTC) Reply

I don't think so. Ideally even what are we using (HAProxy, ATS or varnish) should be irrelevant Vgutierrez ( talk ) 09:39, 17 February 2023 (UTC) Reply
Yes Done BCornwall ( talk ) 16:13, 17 February 2023 (UTC) Reply

What happens when HAProxy closes a connection that's already reached Varnish/ATS?

When talking with cdanis about the rate limiting, he mentioned:

<cdanis> doesn't cancel any of the requests running in Varnish/MW/etc
<brett> So what happens with the requests running in varnish/mw/etc? Are those returned as normal or do they go to a black hole?
<cdanis> brett: I haven't checked but I think haproxy closes the connections with those requests pending; not 100% sure what Varnish does from there, or what happens in the applayer, however I suspect they keep running
<rzl> within MW at least, I'm pretty sure once the request has started, it'll continue until we try to send the first response byte and get an error
<rzl> I wouldn't be surprised if proxy components like Varnish, or even Envoy on the appserver, release their resources for that request more promptly, but I don't know what each of them does
<rzl> (actually I guess "try to send the first response byte and get an error" depends on Envoy's behavior in particular)
<rzl> but MW will proceed, until either TTFB or the entire request, as normal

Is this information that is relevant to this article? BCornwall ( talk ) 20:20, 28 February 2023 (UTC) Reply

Removing individual component detailing in favor of more consumer-based, factual information

Vgutierrez : I got some useful feedback from bblack regarding the overall structure of the article based on the audience:

<bblack> the "audience" for this, I assume, is application-layer folks? this is more or less the start of documenting our "contract" or whatever?
<bblack> in which case, I kind of question the splitting of it by layers in general
<bblack> e.g. the "caching logic" at the bottom under BCL - the FCL has to follow the same rules, too.
<bblack> and the headers set from the TLS and FCL layers, well, it's the BCL that carries them through to the app.
<bblack> yeah, plus long-term-thinking: we may change the number of layers and how they're implemented (and have in the past), but we try to keep the /behavior/ invariant from the outside
<bblack> so maybe just a brief into section that talks about the current structure of things (i.e. that it's currently implemented in 3 layers using haproxy,varnish/ats - because this /is/ useful to know when debugging things or thinking about things in the present).
<bblack> but then talk about headers/protocols/etc for the CDN as a whole (many of these are rules that carry through all layers, or must be implemented in more than one, or may move where they're implemented at in the future)

I think this makes sense and it's worth restructuring this by removing all the software destinction and just say "this is what you will receive from our side of things". BCornwall ( talk ) 18:26, 27 March 2023 (UTC) Reply

+1 :) Vgutierrez ( talk ) 18:36, 27 March 2023 (UTC) Reply
Yes Done - It still needs improvements but at least the structure is now there. BCornwall ( talk ) 21:33, 27 March 2023 (UTC) Reply

Is it time to move this out of the User page?

I don't want it to be forgotten/lost :) BCornwall ( talk ) 20:13, 18 May 2023 (UTC) Reply