Hi all
I'm happy to let you know that new hardware has been ordered by Wikimedia
Deutschland and will arrive probably in about two weeks. We will get two new
systems:
* A more powerful web server, to replace hemlock: Sun Fire X4150, 2x Quad-Core
Xeon, 8GB RAM, 2x73GB SAS HDD. The current web server only has two cores.
* Another database server, to be used for S1 (english wikipedia), so S1 and S3
no longer have to share a server: Sun Fire X4250, 2x Quad-Core Xeon, 32GB RAM,
16x146GB SAS RAID.
This should improve performance and give us some head space for growth. Once the
new servers arrive, S3 will be re-imported too, so we will have live data again.
Any ideas for names? To stay with the nightshade theme, how about Jurubeba and
Erubia? Or perhaps we go the "witches' weed" way, with Datura and Mandrake?
Henbane is taken, i think. Amanita sounds nice, too :)
A third server has been ordered, which will also be installed in Amsterdam, but
will not be part of the toolserver cluster. It's a storage server (X4540, 24TB
RAID) that will keep a live backup of all media files.
Cheers,
Daniel
>> hi all
>>
>> many of our tools uses messages from betawiki because they are
>> translated in many languages.
>> But this messages must be synchronized and I think it's not very
>> economically if every user/tool do that self. Because of that I
>> propose
>> to create one database for all users which will be sync periodically.
>
>A database with one table per user? Sounds good.
>
>> What do you think? Are any other users interested for such a db?
>
>I would be interested.
>
>> Greetings,
>> Luxo
>
>Pietrodn
>powerpdn(a)gmail.com
I thought not one table per user, but rather one table for all messages.
On betawiki are a lot messages e.g.
http://translatewiki.net/wiki/MediaWiki:Abusefilter-revert-title/en.
So I would make a table something like
+
----------+------------------------+-----------------------------------------------------+
| LANG | MESSAGE |
TEXT |
+
----------+------------------------+-----------------------------------------------------+
| en | Abusefilter-revert-title | Revert all changes by
filter $1 |
+
----------+------------------------+-----------------------------------------------------+
| de | Abusefilter-revert-title | Alle Änderungen durch
Filter $1 rückgängig machen |
+
----------+------------------------+-----------------------------------------------------+
| fr | Abusefilter-revert-title | Révoquer toutes les
modifications par le filtre $1 |
+
----------+------------------------+-----------------------------------------------------+
the messages which will be synced should be one a list which every
ts-user can edit, so that it's easy for every user to add the messages
he requires.
--Luxo
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hi,
the new s2 server (daphne) is online, and should have caught up with
replication later today. however, it is using a new MySQL build (version
5.1.35), which contains a critical InnoDB bug causing server crashes when
innodb_locks_unsafe_for_binlog is enabled. since we need this option to
support long queries, we won't switch servers until a patch is available.
(this is a new bug, unrelated to the previous one.) i hope this will happen
later this week.
when we switch, user databases currently on the s2 server will _not_ be moved,
and the 'sql' alias will continue to point at the old server (zedler).
therefore, if you have created tables on the old server which you need to join
to s2 databases, you will need to move these to the new server yourself. do
not move normal user databases, they should remain on 'sql' as before.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (HP-UX)
iEYEARECAAYFAko+R7MACgkQIXd7fCuc5vIV9gCdGvqGabLQCienTQOB3Lk5ClyP
l3EAoIZebLgBV8X6zeUIspNZNn1C8mlE
=uYxB
-----END PGP SIGNATURE-----
Heres a little side project idea for anyone interested:
I often see people wanting dumps of one or two articles from a
Wikipedia wiki (most commonly en) and most commonly have to resort to
screen scraping via third party tools which isn't very nice and some
times even blocked (at the squid level) although there is the API, it
isn't exactly the most easier system for anyone to use. What might be
nice is a little tool where people can enter a few article names in a
box and click a button and have it produce the static html dumps of
the desired article(/s).
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hi,
given the current low uptake of stable server usage, we have decided we cannot
afford to dedicate an entire server to it, which ends up being mostly idle.
therefore we will be migrating the stable server to a zone (virtual machine) on
hyacinth, which is also the new /home server. there should be no change in
either reliability or performance as a result.
unlike the previous move, no user intervention will be required this time. we
are currently copying the existing /users and /projects directories to the new
server. at switchover time, an incremental rsync will be done to copy any
changes since then. you can log into the new server now, using the hostname
"newstable.toolserver.org"; however, i recommend not changing any files, as
they will be overwritten when the next copy is done. if you would prefer to
edit files on the new server, let me know and i will exclude your project from
the automatic copy.
(it may take a while until you can log in, since not all .ssh directories have
been copied yet.)
i *do* recommend doing a quick sanity check of your software on the new server;
for example, that any software or libraries it needs are available and work
correctly.
downtime for the final migration will be on Monday, 6th June, in line with the
maintenance schedule for stable. we might begin the downtime slightly earlier
than usual (5AM UTC) to allow time for the file copies.
i would like to change to ZWS at the same time as the migration; if your
project has no yet confirmed that it works with ZWS, *please* consider trying
to do so before the migration, as it will make things much easier for us. this
applies to:
* wma
* wmfgcbot
if you are unable to complete the web server migration yourself, please ask for
help, either on IRC or the mailing list.
the move is accompanied by a more recent version of the TS package repository
(/opt/ts), which includes newer versions of Perl, Python and other software.
this is documented in more detail at:
<https://wiki.toolserver.org/view/Solaris_software>
for projects using third-party Python modules currently installed in
/opt/ts/lib/python2.4/lib/site-packages, these will still be available after
the move. however, we recommend changing to the Toolserver Python.
after the move is complete, the existing stable server (willow) will be
repurposed into a general login server. more details on this will follow
later.
lastly, we are still interested in more stable projects. stable server
projects can have multiple maintainers, which reduces the workload on one
maintainer, and means tools don't die when their maintainers leave.
we are currently considering disabling the unmaintained web tools of expired
user accounts, meaning these tools will stop working. stable tools will only
be disabled if *all* maintainers leave, and no new maintainer can be found.
additionally, the stable Toolserver has a well-defined maintenance schedule to
avoid downtime during peak times, and has much less unplanned downtime than the
normal Toolserver.
if you would like to move a project to stable, open a request in the TS project
in JIRA, describing the project and listing the initial maintainer(s).
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (HP-UX)
iEYEARECAAYFAko8YXMACgkQIXd7fCuc5vKkogCgkUtkExNsFNbczDCq9d4IbT6v
S0sAmgLHns9mqnkzDwMjkICWpVnNJsI1
=11m+
-----END PGP SIGNATURE-----
Hi all
during wikimedia germany's brithday party yesterday, I was asked to look into
improving the communication between toolserver folks and the wiki communities
using the toolserver. We have become an important part of their infrastructure,
and when things break, they notice - but often don't know who to ask about it.
So, I would like to propose a few things to improve this situation.
1) There should be a short description of the toolserver, with the relevant
links, in the project namespace of each wiki that us the toolserver
significantly. That page should be maintained by the local community of course,
but I think it would be ideal if a toolserver user who is also a member of that
community would help with that, or even start the page. I will do this for the
german wikipedia soon.
2) we, the toolserver admins, should use the toolserver blog more to communicate
what's going on with the TS. I guess several of the messages I posted here
lately should have gone to the blog too. Maybe blog posts could be forwarded to
toolserver-l automatically? posting the same thing to several places is annoying...
3) we should incurage people to describe their tools on the toolserver wiki,
kind of like the description pages for extension on mediawiki.org. Perhaps we
should have a guideline for this?
The idea is to promote some more information about who we are, what we do, and
how the toolserver works. And we should also help people to find the right
person to contact when things break. What do you think?
-- daniel
Hello, I also believe that it would a big benefit if we would have a
better overview over existing tools in the toolserver wiki.
So I created today some pages for my own tools there. Because I had
already project pages for this I only link to them, which wasn't hard work.
The benefit is that we can begin to categories tools by topics. I hope
other developer see also that it is necessary and do the same.
The documentation can avoid that we do things duplicated and you can
find an alternative tool easier.
It would be also good if we can show what we have if we want more money
for better hardware...
In the wiki it would be perhaps good if we would have a "tool"
namespace. So you can better search for tools or showing all tools. Or
should we use the toolserver wiki as semanticwiki-playground ?
Greetings Kolossos
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hi,
the two new Toolservers, hyacinth (/home server) and daphne (s2 database) have
been installed. on Monday, June 15, between 5-7AM UTC, i will move /home from
hemlock to hyacinth; this will involve some downtime as the entire filesystem
has to be copied. the total downtime should be less than 2 hours.
more information will be forthcoming later regarding the new s2 database,
and the OpenStreetMap Toolserver, which was also installed today.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (HP-UX)
iEYEARECAAYFAko0FGwACgkQIXd7fCuc5vIQ8QCdGjAp23Ug0Aa/X+0633PsnUYG
hT4AmQHOnuLToyrUBFTHtPCMLUnH+daQ
=Ch82
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hi,
as usual, the following list contains users whose home directories are using
more than 1GB of disk space. please consider checking your home directory and
deleting any files you no longer need.
space is listed in kilobytes.
1044294 mzmcbride
1077935 leon
1138620 vvv
1231049 tawker
1257963 autocracy
1461143 erenrich
1520995 danny_b
1714427 purodha
1898741 fmaunier
2388170 misza13
2498892 darkdadaah
2620823 flacus
2929883 skyluke
3352872 stwalkerster
4541646 henna
5029182 drh08
5070470 tparscal
5637043 gmaxwell
6229967 kolossos
7113430 werdna
9724094 dschwen
10780027 cbm
15499050 bryan
15866279 emijrp
21621695 voj
33116919 sk
56239790 daniel
117855613 soxred93
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (HP-UX)
iEYEARECAAYFAko2joUACgkQIXd7fCuc5vLVRACfWRMPy/fW4sUApfCS84V0yEgW
Og4AoLJNcVkPjCy8F1RE9KWNVMG8i2G3
=2gev
-----END PGP SIGNATURE-----
Hello all,
I need help. I have a perl programme (for Check Wikipedia). This can
scan a dump of a language very fast. 2200 pages per minute is no problem.
I will daily scan with the same script the page text of the live
Wikipedia. Not all pages, but maybe 20000 per day per language. Normally
this need only 10 minutes with the dump, but with the live Wikipedia
this need many time. I use the Wikipedia-API to get the text of an
article and so my script can only scan 120 pages per minute. So this
scan need at the moment in enwiki 300 minutes or in dewiki 134 minutes.
The most time my script is waiting. This is a problem because there is a
high CPU usage.
I need a faster way to get the text from the live Wikipedia. So I can
reduce the CPU usage.
Maybe someone know a faster way! Or have an other idea.
Thanks for every help,
Stefan (sk)
More Info:
http://de.wikipedia.org/wiki/Benutzer:Stefan_Kühn/Check_Wikipediahttp://en.wikipedia.org/wiki/Wikipedia:WikiProject_Check_Wikipediahttp://toolserver.org/~sk/checkwiki/checkwiki.pl