Skip to content

Commit 8ca8aef

Browse files
fabpotfabpot
committed
merged branch stof/gettext_loader (PR #2412)
Commits ------- d974a4a Merge pull request #4 from stealth35/test_mo_loader cf05646 delete useless tests 19f9de9 [Translation] fix gettext tests 965f2bf Merge pull request #3 from stealth35/test_mo_loader 9c2a26d [Translation] add Mo loader tests 9af2342 [Translation] Added the gettext loaders Discussion ---------- [Translation] Added the gettext loaders This is the squashed version of the work done by @xaav in #634. @stealth35 you said you will work on the dumpers. do you have some stuff on it ? --------------------------------------------------------------------------- by drak at 2011/10/24 19:28:43 -0700 Is there any more progress with this? --------------------------------------------------------------------------- by stealth35 at 2011/10/25 00:57:19 -0700 I work on the dumpers, but the Po loader is wrong, caus' the Po ressource can be multiline, msgid "" "Here is an example of how one might continue a very long string\n" "for the common case the string represents multi-line output.\n" http://www.gnu.org/software/gettext/manual/gettext.html#PO-Files Anyway the Po format is an intermediate format to Mo file, (like .txt to .res file for ICU), IMO we can just support the real gettext format : Mo --------------------------------------------------------------------------- by stealth35 at 2011/11/03 02:00:24 -0700 @stof The MO Dumper is ready (stealth35/symfony@f2d1d5b), should we keep the PO format ? --------------------------------------------------------------------------- by fabpot at 2011/11/07 08:50:59 -0800 @stealth35: The PO is what people will use for their translations. They will then dump it to MO. So, we need both PO and MO loaders and dumpers. --------------------------------------------------------------------------- by stealth35 at 2011/11/08 01:25:39 -0800 @fabpot, I'm ready for both dumpers, you can merge this, and I'll open a PR for the dumpers --------------------------------------------------------------------------- by fabpot at 2011/11/08 22:37:47 -0800 I've just had a look at this PR code again and I see that the unit tests are pretty slim. Is it possible to add some tests for the mo loader? --------------------------------------------------------------------------- by stealth35 at 2011/11/09 01:15:25 -0800 @fabpot test send to @stof ✌️ --------------------------------------------------------------------------- by stof at 2011/11/09 02:22:55 -0800 and merged in this branch --------------------------------------------------------------------------- by fabpot at 2011/11/09 02:39:09 -0800 The tests do not pass for me: There was 1 error: 1) Symfony\Tests\Component\Translation\Loader\MoFileLoaderTest::testLoadDoesNothingIfEmpty InvalidArgumentException: MO stream content has an invalid format. /Users/fabien/work/symfony/git/symfony/src/Symfony/Component/Translation/Loader/MoFileLoader.php:79 /Users/fabien/work/symfony/git/symfony/src/Symfony/Component/Translation/Loader/MoFileLoader.php:46 /Users/fabien/work/symfony/git/symfony/tests/Symfony/Tests/Component/Translation/Loader/MoFileLoaderTest.php:34 -- There was 1 failure: 1) Symfony\Tests\Component\Translation\Loader\PoFileLoaderTest::testLoad Failed asserting that two arrays are equal. --- Expected +++ Actual @@ @@ Array ( - 'foo' => 'bar' ) /Users/fabien/work/symfony/git/symfony/tests/Symfony/Tests/Component/Translation/Loader/PoFileLoaderTest.php:25
2 parents ad6ffea + d974a4a commit 8ca8aef

File tree

9 files changed

+365
-0
lines changed

9 files changed

+365
-0
lines changed

‎src/Symfony/Bundle/FrameworkBundle/Resources/config/translation.xml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@
1111
<parameter key="translation.loader.php.class">Symfony\Component\Translation\Loader\PhpFileLoader</parameter>
1212
<parameter key="translation.loader.yml.class">Symfony\Component\Translation\Loader\YamlFileLoader</parameter>
1313
<parameter key="translation.loader.xliff.class">Symfony\Component\Translation\Loader\XliffFileLoader</parameter>
14+
<parameter key="translation.loader.po.class">Symfony\Component\Translation\Loader\PoFileLoader</parameter>
15+
<parameter key="translation.loader.mo.class">Symfony\Component\Translation\Loader\MoFileLoader</parameter>
1416
<parameter key="translation.loader.qt.class">Symfony\Component\Translation\Loader\QtTranslationsLoader</parameter>
1517
<parameter key="translation.loader.csv.class">Symfony\Component\Translation\Loader\CsvFileLoader</parameter>
1618
<parameter key="translation.loader.rb.class">Symfony\Component\Translation\Loader\ResourceBundleLoader</parameter>
@@ -56,6 +58,14 @@
5658
<tag name="translation.loader" alias="xliff" />
5759
</service>
5860

61+
<service id="translation.loader.po" class="%translation.loader.po.class%">
62+
<tag name="translation.loader" alias="po" />
63+
</service>
64+
65+
<service id="translation.loader.mo" class="%translation.loader.mo.class%">
66+
<tag name="translation.loader" alias="mo" />
67+
</service>
68+
5969
<service id="translation.loader.qt" class="%translation.loader.qt.class%">
6070
<tag name="translation.loader" alias="ts" />
6171
</service>

‎src/Symfony/Component/Translation/Loader/MoFileLoader.php

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Translation\Loader;
13+
14+
use Symfony\Component\Config\Resource\FileResource;
15+
16+
/**
17+
* @copyright Copyright (c) 2010, Union of RAD http://union-of-rad.org (http://lithify.me/)
18+
*/
19+
class MoFileLoader extends ArrayLoader implements LoaderInterface
20+
{
21+
/**
22+
* Magic used for validating the format of a MO file as well as
23+
* detecting if the machine used to create that file was little endian.
24+
*
25+
* @var float
26+
*/
27+
const MO_LITTLE_ENDIAN_MAGIC = 0x950412de;
28+
29+
/**
30+
* Magic used for validating the format of a MO file as well as
31+
* detecting if the machine used to create that file was big endian.
32+
*
33+
* @var float
34+
*/
35+
const MO_BIG_ENDIAN_MAGIC = 0xde120495;
36+
37+
/**
38+
* The size of the header of a MO file in bytes.
39+
*
40+
* @var integer Number of bytes.
41+
*/
42+
const MO_HEADER_SIZE = 28;
43+
44+
public function load($resource, $locale, $domain = 'messages')
45+
{
46+
$messages = $this->parse($resource);
47+
48+
// empty file
49+
if (null === $messages) {
50+
$messages = array();
51+
}
52+
53+
// not an array
54+
if (!is_array($messages)) {
55+
throw new \InvalidArgumentException(sprintf('The file "%s" must contain a valid mo file.', $resource));
56+
}
57+
58+
$catalogue = parent::load($messages, $locale, $domain);
59+
$catalogue->addResource(new FileResource($resource));
60+
61+
return $catalogue;
62+
}
63+
64+
/**
65+
* Parses machine object (MO) format, independent of the machine's endian it
66+
* was created on. Both 32bit and 64bit systems are supported.
67+
*
68+
* @param resource $stream
69+
* @return array
70+
* @throws InvalidArgumentException If stream content has an invalid format.
71+
*/
72+
private function parse($resource)
73+
{
74+
$stream = fopen($resource, 'r');
75+
76+
$stat = fstat($stream);
77+
78+
if ($stat['size'] < self::MO_HEADER_SIZE) {
79+
throw new \InvalidArgumentException("MO stream content has an invalid format.");
80+
}
81+
$magic = unpack('V1', fread($stream, 4));
82+
$magic = hexdec(substr(dechex(current($magic)), -8));
83+
84+
if ($magic == self::MO_LITTLE_ENDIAN_MAGIC) {
85+
$isBigEndian = false;
86+
} elseif ($magic == self::MO_BIG_ENDIAN_MAGIC) {
87+
$isBigEndian = true;
88+
} else {
89+
throw new \InvalidArgumentException("MO stream content has an invalid format.");
90+
}
91+
92+
$header = array(
93+
'formatRevision' => null,
94+
'count' => null,
95+
'offsetId' => null,
96+
'offsetTranslated' => null,
97+
'sizeHashes' => null,
98+
'offsetHashes' => null,
99+
);
100+
foreach ($header as &$value) {
101+
$value = $this->readLong($stream, $isBigEndian);
102+
}
103+
extract($header);
104+
$messages = array();
105+
106+
for ($i = 0; $i < $count; $i++) {
107+
$singularId = $pluralId = null;
108+
$translated = null;
109+
110+
fseek($stream, $offsetId + $i * 8);
111+
112+
$length = $this->readLong($stream, $isBigEndian);
113+
$offset = $this->readLong($stream, $isBigEndian);
114+
115+
if ($length < 1) {
116+
continue;
117+
}
118+
119+
fseek($stream, $offset);
120+
$singularId = fread($stream, $length);
121+
122+
if (strpos($singularId, "\000") !== false) {
123+
list($singularId, $pluralId) = explode("\000", $singularId);
124+
}
125+
126+
fseek($stream, $offsetTranslated + $i * 8);
127+
$length = $this->readLong($stream, $isBigEndian);
128+
$offset = $this->readLong($stream, $isBigEndian);
129+
130+
fseek($stream, $offset);
131+
$translated = fread($stream, $length);
132+
133+
if (strpos($translated, "\000") !== false) {
134+
$translated = explode("\000", $translated);
135+
}
136+
137+
$ids = array('singular' => $singularId, 'plural' => $pluralId);
138+
$item = compact('ids', 'translated');
139+
140+
if (is_array($item['translated'])) {
141+
$messages[$item['ids']['singular']] = stripslashes($item['translated'][0]);
142+
if (isset($item['ids']['plural'])) {
143+
$messages[$item['ids']['plural']] = stripslashes(end($item['translated']));
144+
}
145+
} elseif($item['ids']['singular']) {
146+
$messages[$item['ids']['singular']] = stripslashes($item['translated']);
147+
}
148+
}
149+
150+
fclose($stream);
151+
152+
return array_filter($messages);
153+
}
154+
155+
/**
156+
* Reads an unsigned long from stream respecting endianess.
157+
*
158+
* @param resource $stream
159+
* @param boolean $isBigEndian
160+
* @return integer
161+
*/
162+
private function readLong($stream, $isBigEndian)
163+
{
164+
$result = unpack($isBigEndian ? 'N1' : 'V1', fread($stream, 4));
165+
$result = current($result);
166+
167+
return (integer) substr($result, -8);
168+
}
169+
}

‎src/Symfony/Component/Translation/Loader/PoFileLoader.php

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Translation\Loader;
13+
14+
use Symfony\Component\Config\Resource\FileResource;
15+
16+
/**
17+
* @copyright Copyright (c) 2010, Union of RAD http://union-of-rad.org (http://lithify.me/)
18+
*/
19+
class PoFileLoader extends ArrayLoader implements LoaderInterface
20+
{
21+
public function load($resource, $locale, $domain = 'messages')
22+
{
23+
$messages = $this->parse($resource);
24+
25+
// empty file
26+
if (null === $messages) {
27+
$messages = array();
28+
}
29+
30+
// not an array
31+
if (!is_array($messages)) {
32+
throw new \InvalidArgumentException(sprintf('The file "%s" must contain a valid po file.', $resource));
33+
}
34+
35+
$catalogue = parent::load($messages, $locale, $domain);
36+
$catalogue->addResource(new FileResource($resource));
37+
38+
return $catalogue;
39+
}
40+
41+
/**
42+
* Parses portable object (PO) format.
43+
*
44+
* This parser sacrifices some features of the reference implementation the
45+
* differences to that implementation are as follows.
46+
* - No support for comments spanning multiple lines.
47+
* - Translator and extracted comments are treated as being the same type.
48+
* - Message IDs are allowed to have other encodings as just US-ASCII.
49+
*
50+
* Items with an empty id are ignored.
51+
*
52+
* @param resource $stream
53+
* @return array
54+
*/
55+
private function parse($resource)
56+
{
57+
$stream = fopen($resource, 'r');
58+
59+
$defaults = array(
60+
'ids' => array(),
61+
'translated' => null,
62+
);
63+
64+
$messages = array();
65+
$item = $defaults;
66+
67+
while ($line = fgets($stream)) {
68+
$line = trim($line);
69+
70+
if ($line === '') {
71+
if (is_array($item['translated'])) {
72+
$messages[$item['ids']['singular']] = stripslashes($item['translated'][0]);
73+
if (isset($item['ids']['plural'])) {
74+
$messages[$item['ids']['plural']] = stripslashes(end($item['translated']));
75+
}
76+
} elseif($item['ids']['singular']) {
77+
$messages[$item['ids']['singular']] = stripslashes($item['translated']);
78+
}
79+
$item = $defaults;
80+
} elseif (substr($line, 0, 7) === 'msgid "') {
81+
$item['ids']['singular'] = substr($line, 7, -1);
82+
} elseif (substr($line, 0, 8) === 'msgstr "') {
83+
$item['translated'] = substr($line, 8, -1);
84+
} elseif ($line[0] === '"') {
85+
$continues = isset($item['translated']) ? 'translated' : 'ids';
86+
87+
if (is_array($item[$continues])) {
88+
end($item[$continues]);
89+
$item[$continues][key($item[$continues])] .= substr($line, 1, -1);
90+
} else {
91+
$item[$continues] .= substr($line, 1, -1);
92+
}
93+
} elseif (substr($line, 0, 14) === 'msgid_plural "') {
94+
$item['ids']['plural'] = substr($line, 14, -1);
95+
} elseif (substr($line, 0, 7) === 'msgstr[') {
96+
$item['translated'][(integer) substr($line, 7, 1)] = substr($line, 11, -1);
97+
}
98+
99+
}
100+
fclose($stream);
101+
102+
return array_filter($messages);
103+
}
104+
}

‎tests/Symfony/Tests/Component/Translation/Loader/MoFileLoaderTest.php

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Tests\Component\Translation\Loader;
13+
14+
use Symfony\Component\Translation\Loader\MoFileLoader;
15+
use Symfony\Component\Config\Resource\FileResource;
16+
17+
class MoFileLoaderTest extends \PHPUnit_Framework_TestCase
18+
{
19+
public function testLoad()
20+
{
21+
$loader = new MoFileLoader();
22+
$resource = __DIR__.'/../fixtures/resources.mo';
23+
$catalogue = $loader->load($resource, 'en', 'domain1');
24+
25+
$this->assertEquals(array('foo' => 'bar'), $catalogue->all('domain1'));
26+
$this->assertEquals('en', $catalogue->getLocale());
27+
$this->assertEquals(array(new FileResource($resource)), $catalogue->getResources());
28+
}
29+
30+
/**
31+
* @expectedException \InvalidArgumentException
32+
*/
33+
public function testLoadInvalidResource()
34+
{
35+
$loader = new MoFileLoader();
36+
$resource = __DIR__.'/../fixtures/empty.mo';
37+
$catalogue = $loader->load($resource, 'en', 'domain1');
38+
}
39+
}

‎tests/Symfony/Tests/Component/Translation/Loader/PoFileLoaderTest.php

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Tests\Component\Translation\Loader;
13+
14+
use Symfony\Component\Translation\Loader\PoFileLoader;
15+
use Symfony\Component\Config\Resource\FileResource;
16+
17+
class PoFileLoaderTest extends \PHPUnit_Framework_TestCase
18+
{
19+
public function testLoad()
20+
{
21+
$loader = new PoFileLoader();
22+
$resource = __DIR__.'/../fixtures/resources.po';
23+
$catalogue = $loader->load($resource, 'en', 'domain1');
24+
25+
$this->assertEquals(array('foo' => 'bar'), $catalogue->all('domain1'));
26+
$this->assertEquals('en', $catalogue->getLocale());
27+
$this->assertEquals(array(new FileResource($resource)), $catalogue->getResources());
28+
}
29+
30+
public function testLoadDoesNothingIfEmpty()
31+
{
32+
$loader = new PoFileLoader();
33+
$resource = __DIR__.'/../fixtures/empty.po';
34+
$catalogue = $loader->load($resource, 'en', 'domain1');
35+
36+
$this->assertEquals(array(), $catalogue->all('domain1'));
37+
$this->assertEquals('en', $catalogue->getLocale());
38+
$this->assertEquals(array(new FileResource($resource)), $catalogue->getResources());
39+
}
40+
}

‎tests/Symfony/Tests/Component/Translation/fixtures/empty.mo

Whitespace-only changes.

‎tests/Symfony/Tests/Component/Translation/fixtures/empty.po

Whitespace-only changes.

‎tests/Symfony/Tests/Component/Translation/fixtures/resources.mo

52 Bytes
Binary file not shown.

‎tests/Symfony/Tests/Component/Translation/fixtures/resources.po

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
msgid "foo"
2+
msgstr "bar"
3+

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy