TLD cvsup mirrors

[Ken Smith]

I'm looking for several new cvsup mirrors for the top level domain
(cvsup*.freebsd.org). My preference is to "promote" sites that have
been working for a while as sites within a country code that feel they
can take on more traffic. At this point most of the TLD cvsup servers
are in the US (and not to put too fine a point on it a depressing number
are in buffalo.edu because I "temporarily" took over for another server
that went AWOL and I'm just now getting around to looking for a real
replacement...) so sites in other countries are definitely encouraged to
make an offer. I'd like to diversify it a bit if possible.

Just to give you a feel for what you'd be getting into... Some of these
sites currently handle as many as 1000 to 2000 connects/day. From my
experiences if you want to support that many connects/day in a
reasonable fashion it takes a machine that can handle 20 to 30
concurrent connections. To do that it helps to have learned a few
tricks like making sure to turn off atime on the filesystem the
repository is in, and bumping up kern.maxvnodes to encourage more file
caching.

Thanks.

[Adam McDougall]

Post by Ken Smith
I'm looking for several new cvsup mirrors for the top level domain
(cvsup*.freebsd.org). My preference is to "promote" sites that have
been working for a while as sites within a country code that feel they
can take on more traffic. At this point most of the TLD cvsup servers
are in the US (and not to put too fine a point on it a depressing number
are in buffalo.edu because I "temporarily" took over for another server
that went AWOL and I'm just now getting around to looking for a real
replacement...) so sites in other countries are definitely encouraged to
make an offer. I'd like to diversify it a bit if possible.
Just to give you a feel for what you'd be getting into... Some of these
sites currently handle as many as 1000 to 2000 connects/day. From my
experiences if you want to support that many connects/day in a
reasonable fashion it takes a machine that can handle 20 to 30
concurrent connections. To do that it helps to have learned a few
tricks like making sure to turn off atime on the filesystem the
repository is in, and bumping up kern.maxvnodes to encourage more file
caching.
Thanks.

I'd be happy to take on more cvsupd clients if you want, lately I've
been getting around 550 working clients per day and it feels like
nothing (cvsup15.freebsd.org). I have 20G of ram in the system and
lately a lot of the cvsup data seems to get cached. Plenty of CPU to go
around too.

What would you recommend for kern.maxvnodes? Thanks

[Ken Smith]

Post by Adam McDougall
I'd be happy to take on more cvsupd clients if you want, lately I've
been getting around 550 working clients per day and it feels like
nothing (cvsup15.freebsd.org). I have 20G of ram in the system and
lately a lot of the cvsup data seems to get cached. Plenty of CPU to go
around too.

Thanks. If I don't get enough offers I might take you up on that but at
least for now the goal is to get back to having each of the
cvsup*.freebsd.org entries being unique machines. The higher numbered
machines do seem to wind up with less traffic, one of mine is cvsup18
and it only sees around 200/day.

Post by Adam McDougall
What would you recommend for kern.maxvnodes? Thanks

That depends on how much physical memory the machine has and what else
it's doing. For my machines, which are just mirror systems, I have:

Dual Opteron 2.3GHz, 4GB RAM:
kern.maxvnodes = 250000
maximum cvsupd clients = 40

Quad Xeon 2.4GHZ, 8GB RAM:
kern.maxvnodes = 350000
maximum cvsupd clients = 40

Dual Xeon 1.7GHz, 2GB RAM:
kern.maxvnodes = 150000
maximum cvsupd clients = 20

The second one is ftp-archive.freebsd.org as well as being a cvsupd
server.

With 20GB RAM if the machine isn't doing much else you could probably
crank maxvnodes up to over the number of files in the repository itself
(looks to be a little over 570000 files in it at the moment).

The other performance tweak that helps a *lot* is to make sure you mount
the partition the repository is in using noatime.

The first machine listed above handles around 3000 connects per day
comfortably with that configuration. It's the one I let accumulate a
few too many of the TLD names though (three at the moment), those are
some of what I'm trying to find a new home for. :-)

I only have the second machine doing significant-ish cvsupd stuff when I
need it to take over for site(s) that died while I look to find a new
site to take it over. If a cvsupd server has been down for a while it
can wind up getting *slammed* when it comes back to life. I took over
for one that had been down for quite a while on Friday and it wound up
handling over 7,000 requests the first day. So... I usually try to take
over on one of my machines first to get through that and then get an
idea of what the normal daily traffic is so I can tell the new site what
they're in for...

[Garrett Wollman]

Post by Ken Smith
Thanks. If I don't get enough offers I might take you up on that but at
least for now the goal is to get back to having each of the
cvsup*.freebsd.org entries being unique machines. The higher numbered
machines do seem to wind up with less traffic, one of mine is cvsup18
and it only sees around 200/day.

cvsup3 gets about 600 clients/day (averaged over the past week) with a
connection limit of 16.

-GAWollman

[Rutger Bevaart]

I'd be happy to add our CVSup machine to the list, being located in
the Netherlands. To me it seems a lot of cvsup mirrors in the
Netherlands are aliases of the one located at SURFnet.

Anybody on the list who can check/test and add ours?

cvsup.illian.net located in AS39292.

Met vriendelijke groet / Kind Regards,

Rutger Bevaart

illian.networks B.V.

Costerweg 5
6702AA Wageningen

T: 0317.712.400
F: 0317.712.401
W: http://www.illian-networks.nl

Post by Garrett Wollman

Post by Ken Smith
Thanks. If I don't get enough offers I might take you up on that but at
least for now the goal is to get back to having each of the
cvsup*.freebsd.org entries being unique machines. The higher
numbered
machines do seem to wind up with less traffic, one of mine is cvsup18
and it only sees around 200/day.

cvsup3 gets about 600 clients/day (averaged over the past week) with a
connection limit of 16.
-GAWollman
_______________________________________________
http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
To unsubscribe, send any mail to "freebsd-hubs-

[Mark Huizer]

Post by Rutger Bevaart
I'd be happy to add our CVSup machine to the list, being located in
the Netherlands. To me it seems a lot of cvsup mirrors in the
Netherlands are aliases of the one located at SURFnet.
Anybody on the list who can check/test and add ours?
cvsup.illian.net located in AS39292.

Actually, they are aliases of a machine that is not at surfnet but at
ClaraNet Netherlands. :-)
But if you want to be added as cvsup2.nl.freebsd.org, be my guest, just
contact me at hostmaster at nl.freebsd.org

Greetings

Mark

[ben wilber]

Post by Ken Smith
Thanks. If I don't get enough offers I might take you up on that but at
least for now the goal is to get back to having each of the
cvsup*.freebsd.org entries being unique machines. The higher numbered
machines do seem to wind up with less traffic, one of mine is cvsup18
and it only sees around 200/day.

If you need any more US-based servers, I volunteer cvsup5.us. It gets
~500 connections per day and runs basically idle.

bw.