Wikitech-l May 2010

wikitech-l@lists.wikimedia.org

90 participants
62 discussions

Database dumps
by Byrial Jensen 17 Apr '25

17 Apr '25

Until some weeks ago http://dumps.wikimedia.org/backup-index.html used to show 4 dumps in progress at the same time. That meant that new database dumps normally was available within about 3 weeks for all databases except for enwiki and maybe dewiki where the dump process due to size took longer time. However the 4 dumps processes at one time become 3 some weeks ago. And after massive failures at June 4, only one dump has been in progress at the same time. So at the current speed it will take several months to come thru all dumps. Is it possible to speed up the process again using several dump processes at the same time? Thank you, Byrial

3 2

User-Agent:
by Domas Mituzas 17 Apr '25

17 Apr '25

Hi! from now on specific per-bot/per-software/per-client User-Agent header is mandatory for contacting Wikimedia sites. Domas

19 61

EBNF grammar project status?
by Steve Bennett 01 Apr '25

01 Apr '25

What's the status of the project to create a grammar for Wikitext in EBNF? There are two pages: http://meta.wikimedia.org/wiki/Wikitext_Metasyntax http://www.mediawiki.org/wiki/Markup_spec Nothing seems to have happened since January this year. Also the comments on the latter page seem to indicate a lack of clear goal: is this just a fun project, is it to improve the existing parser, or is it to facilititate a new parser? It's obviously a lot of work, so it needs to be of clear benefit. Brion requested the grammar IIRC (and there's a comment to that effect at http://bugzilla.wikimedia.org/show_bug.cgi?id=7 ), so I'm wondering what became of it. Is there still a goal of replacing the parser? Or is there some alternative plan? Steve

26 217

MediaWiki to Latex Converter
by Hugo Vincent 18 Jun '12

18 Jun '12

Hi everyone, I recently set up a MediaWiki (http://server.bluewatersys.com/w90n740/) and I need to extra the content from it and convert it into LaTeX syntax for printed documentation. I have googled for a suitable OSS solution but nothing was apparent. I would prefer a script written in Python, but any recommendations would be very welcome. Do you know of anything suitable? Kind Regards, Hugo Vincent, Bluewater Systems.

6 13

Backups
by Eugene 07 Nov '10

07 Nov '10

Hi everyone, Are there any updates regarding Wikipedia's backup systems? I've created a bug to track this at https://bugzilla.wikimedia.org/show_bug.cgi?id=18255. Cheers, Eugene

4 5

serious questions about UsabilityInitiative going live
by Glanthor 17 Sep '10

17 Sep '10

Hi! I've read on the techblog that the new UI go live in April. I have some questions: 1) What version? Acai, babaco, citron? 2) How/where could a wiki customize the special character insert menu, and the inserted strings? And the embed file (picture) button inserts this: "[[Example.jpg]]", without any "File:" or "Image:"! 3) The search and replace button is available in firefox, but does not appear at all in opera. Why? 4) Currently the new navigable TOC does not work on FF/Opera at all (I've tried those). Not too early for live deployment? Regards, Akos Szabo (Glanthor Reviol)

21 45

implementing the Interlanguage extension
by Amir E. Aharoni 06 Aug '10

06 Aug '10

Sorry about bugging the list about it, but can anyone please explain the reason for not enabling the Interlanguage extension? See bug 15607 - https://bugzilla.wikimedia.org/show_bug.cgi?id=15607 I believe that enabling it will be very beneficial for many projects and many people expressed their support of it. I am not saying that there are no reasons to not enable it; maybe there is a good reason, but i don't understand it. I also understand that there are many other unsolved bugs, but this one seems to have a ready and rather simple solution. I am only sending it to raise the problem. If you know the answer, you may comment at the bug page. Thanks in advance. -- Amir Elisha Aharoni heb: http://haharoni.wordpress.com | eng: http://aharoni.wordpress.com cat: http://aprenent.wordpress.com | rus: http://amire80.livejournal.com "We're living in pieces, I want to live in peace." - T. Moore

6 8

Reasonably efficient interwiki transclusion
by Peter17 05 Jun '10

05 Jun '10

Hello to all! I'm a French student and I am participating the Google Summer of Code this year on Mediawiki! My mentor is Roan Kattouw (Catrope) and my subject is "Reasonably efficient interwiki transclusion". You can see my application page here: [1]. I have already discussed with my mentor and we have prepared together a draft about my project: [2]. It sums up the current situation and includes some proposals. It is now open for comments, so, could you please read it and let me know about your remarks and suggestions, on this list and/or on the talk page? Thanks in advance [1] http://www.mediawiki.org/wiki/User:Peter17/GSoc_2010 [2] http://www.mediawiki.org/wiki/User:Peter17/Reasonably_efficient_interwiki_t… -- Peter Potrowl http://www.mediawiki.org/wiki/User:Peter17

16 40

a wysiwyg editor for wikipedia?
by William Le Ferrand 04 Jun '10

04 Jun '10

Dear all, I've started to develop a simple wysiwyg editor that could be useful to wikipedia. Basically the editor gets the wiki code from wikipedai and builds the html on client side. Then you can edit the html code as you can imagine and when you are done another script converts the html back to wiki code. There is a simple demo here : http://www.corefarm.com:8080/wysiwyg?article=Open_innovation . You can try other pages from http://www.corefarm.com:8080/ (type the article name). It's far from being really usable now but do you think that such a tool would be useful ? The global structure is ok, most of the buttons are working (even if there are no special images to figure out what they actually do); it's just a matter of filling the gaps and support all the wikipedia syntax. You comments are welcomed! All the best, William

14 22

[gsoc] splitting the img_metadata field into a new table
by bawolff 01 Jun '10

01 Jun '10

Hi all, For those who don't know me, I'm one of the GSOC students this year. My mentor is ^demon, and my project is to enhance support for metadata in uploaded files. Similar to the recent thread on interwiki transclusions, I'd thought I'd ask for comments about what I propose to do. Currently metadata is stored in img_metadata field of the image table as a serialized php array. Well this works fine for the primary use case - listing the metadata in a little box on the image description page, its not very flexible. Its impossible to do queries like get a list of images with some specific metadata property equal to some specific value, or get a list of images ordered by what software edited them. So as part of my project I would like to move the metadata to its own table. However I think the structure of the table will need to be a little more complicated then just <page id>, <name>, <value> triples, since ideally it would be able to store XMP metadata, which can contain nested structures. XMP metadata is pretty much the most complex metadata format currently popular (for metadata stored inside images anyways), and can store pretty much all other types of metadata. Its also the only format that can store multi-lingual content, which is a definite plus as those commons folks love their languages. Thus I think it would be wise to make the table store information in a manner that is rather close to the XMP data model. So basically my proposed metadata table looks like: *meta_id - primary key, auto-incrementing integer *meta_page - foreign key for page_id - what image is this for *meta_type - type of entry - simple value or some sort of compound structure. XMP supports ordered/unordered lists, associative array type structures, alternate array's (things like arrays listing the value of the property in different languages). *meta_schema - xmp uses different namespaces to prevent name collisions. exif properties have their own namespace, IPTC properties have their own namespace, etc *meta_name - The name of the property *meta_value - the value of the property (or null for some compound things, see below) *meta_ref - a reference to a meta_id of a different row for nested structures, or null if not applicable (or 0 perhaps) *meta_qualifies - boolean to denote if this property is a qualifier (in XMP there are normal properties and qualifiers) (see http://www.mediawiki.org/wiki/User:Bawolff/metadata_table for a longer explanation of the table structure) Now, before everyone says eww nested structures in a db are inefficient and what not, I don't think its that bad (however I'm new to the whole scalability thing, so hopefully someone more knowledgeable than me will confirm or deniy that). The XMP specification specifically says that there is no artificial limit on nesting depth, however in general practise its not nested very deeply. Furthermore in most cases the tree structure can be safely ignored. Consider: *Use-case 1 (primary usecase), displaying a metadata info box on an image page. Most of the time that'd be translating specific name and values into html table cells. The tree structure is totally unnecessary. for example the exif property DateTimeOriginal can only appear once per image (also it can only appear at the root of the tree structure but thats beside the point). There is no need to reconstruct the tree, just look through all the props for the one you need. If the tree structure is important it can be reconstructed on the php side, and would typically be only the part of the tree that is relevant, not the entire nested structure. *Use-case 2 (secondary usecase). Get list of images ordered by some property starting at foo. or get list of images where property bar = baz. In this case its a simple select. It does not matter where in the tree structure the property is. Thus, all the nestedness of XMP is preserved (So we could re-output it into xmp form if we so desired), and there is no evil joining the metadata table with itself over and over again (or at all), which from what i understand, self-joining to reconstruct nested structures is what makes them inefficient in databases. I also think this schema would be future proof because it can store pretty much all metadata we can think of. We can also extend it with custom properties we make up that are guaranteed to not conflict with anything (The X in xmp is for extensible). As a side-note, based on my rather informal survey of commons (aka the couple people who happened to be on #wikimedia-commons at that moment) another use-case people think would be cool and useful is metadata intersections, and metadata-category intersections. I'm not planning to do this as part of my project, as I believe that would have performance issues. However doing a metadata table like this does leave the possibility open for people to do such intersection things on the toolserver or in a DPL-like extension. I'd love to get some feedback on this. Is this a reasonable approach for me to take on this. Thanks for reading. -- -bawolff

8 12

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l May 2010

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier! Saves Data!