Debug your PHP segmentation fault on Travis-CI

Posted on by
Reply

Travis-CI is a powerful tool for continuous integration that I now use daily since few months.

Recently, I experienced many PHP segmentation fault on Travis-CI ; on Symfony 2 Framework and Phraseanet. These segfaults were very difficult to reproduce ; sometimes it was only happening on Travis.

Here’s an how-to run gdb on your PHP build on Travis-CI :

First, create a dedicated branch from where your build is stuck :

git checkout -b GDB

Then, add a gdb script containing you gdb commands, for example :

set env MALLOC_CHECK_=3
run
backtrace full

Now we have to run this gdb script through travis, let’s write a .travis.yml configuration like :

language : php

before_script:
    - sudo apt-get install gdb

script:
    - cat script.gdb | sudo gdb --args /home/travis/.phpenv/versions/5.3.17/bin/php /home/travis/.phpenv/versions/5.3.17/bin/phpunit

As you can read, this build would be run against PHP 5.3.17. You could run it against 5.4.7 by replacing the version number (or any other PHP version available on Travis-CI). Please note that travis version constraints are not valuable anymore with this script.

As a result, you will get an awesome stacktrace that will be very helpfull to PHP core developers to fix it.

Build latest Sphinx Search Engine

Posted on by
Reply

I’ve been working with Sphinx Search Engine for more than a year now.

Sphinx Search (do not confuse with Sphinx Python documentation generator) is an extremely fast full-text search engine designed to index relationnal databases such as MySQL or PostgreSQL CraigsList and Dailymotion use several Sphinx Search boxes for their public search apis. It includes lemmatization and a powerful API including a MySQL connector.
There’s no built in faceted search, but you can build your own as the API allows multiple queries at once (see this article).

As I’m currently improving my knowledge in search engines, I had to build latest Sphinx Search version.

With the latest 5.3 and 5.4 version of PHP, you’ll get an error at connection if you use RealTime indexes (rt indexes) or the MySQL connector with raw Sphinx QL :

PDO::__construct(): Server sent charset (0) unknown to the client. Please, report to the developers

This bug seems to be in PDO as long as mysqli. Temporary workaround is to edit Sphinx Search sources, and patch searchd.cpp :

Find
“\x00″ // server language
and replace by
“\x21″ // server language

This will tell your MySQL client that the emulated Sphinx Search server communicate in utf8.

For those of you who will try to compile on OSX platform, you’ll probably have to deal with a bug in RealTime index (again).
The bug makes RT unusable as it triggers kernel panic. The problem is described in their bug tracker and seems to appears from Sphinx Search 2.0.2 to Sphinx Search 2.0.5. There’s currently no workaround, so you’ll have to use version 2.0.1 available in the download archive for the moment.

Playing with ExactImage

Posted on by
Reply

Recently, I’ve been contributing to Imagine a lot. Imagine is a PHP Image Processing library. It provide an OO abstract layer for processing image with the driver of your choice (Graphics Magick, ImageMagick or GD). The code is quite cool, you should have a look to it.

As I’m having heavy use of this library and was missing some features, I’ve dug the code and I’m now contributing a lot. One of the feature I’m currently working on is ExactImage support. Although ExactImage PHP bindings have been used in production at Skyrock, I didn’t know this lib before somebody asked for it on GitHub and it seems the internet does not know so much about it.

ExactImage is an opensource library developed in Germany by a R&D company focused on image/audio digital signal processing. It has an ImageMagick-like command line API and has been designed for speed.

“In the mid-2000s we were wondering about the ridiculous amounts of CPU cycles ImageMagick spent on certain image processing tasks. After the first code review it became clear that imagemagick was coded for anything but speed”

It uses Swig to generate binding for Perl, Lua, Ruby, Python, PHP. The PHP API is not documented online at all. That’s probably why it’s not used so much.

I’ve posted on GitHub a beginning of the ExactImage PHP API doc. Use it, fork it, share it ! It’s open source, man !

Bindings

Here are some examples of common image processing compared to Imagick API :

Image load

// ImageMagick
$imagick = new \Imagick();
$imagick->readImageFile(file_get_contents('image.jpg'));

// ExactImage
$image = newImage();
decodeImage($image, file_get_contents("image.jpg"));

Image crop

// ImageMagick
$imagick->cropImage($width, $height, $x, $y);

// ExactImage
imageCrop($image, $x, $y, $width, $height);

Resize

// ImageMagick
$imagick->resizeImage(320, 240,
              \Imagick::FILTER_UNDEFINED, 1);

// ExactImage
imageResize($image,320, 240)

Rotation

// ImageMagick
$imagick->rotateimage(
                new \ImagickPixel('fff'), 90);

// ExactImage
setbackgroundcolor(1.0, 1.0, 1.0, 1.0);
imageRotate($image, 90);

Save

// ImageMagick
$imagick->writeImage('modified-image.jpg');

// ExactImage
encodeImageFile($image, 'modified-image.jpg')

Be carefull

Swig provides the PHP extension, defining the same functions as in the original C library. When you call a PHP function, Swig looks for a match between the function you’re calling and its signature (ie the type of the parameters you passed) and the declared functions of ExactImage. For some functions, if you do not use exactly right types, it will result in a fatal error. For example, using setBackgroundColor with integer will produce :

PHP Fatal error: No matching function for overloaded
setBackgroundColor

setBackgroundColor MUST be used with float values :

void setBackgroundColor(float $red,
               float $green, float $blue, float $alpha)

This is unusual in PHP, and I’ve spent some time fighting with the library before having realized that : )

Single drawback

One drawback to exactimage : It does not prefix its functions, and, as it declares functions with the same name as GD (imagerotate), you can not load both extensions at the same time.

More infos :
http://www.exactcode.com/site/open_source/exactimage/
http://www.perlmonks.org/?node_id=705531

Edit 1 :
See doxygen documentation http://fossies.org/dox/exact-image-0.8.7

 

 

I’m having problem with image cache freshness

Posted on by
Reply

Recently, I added consistency and HTTP 1.1 features to Phraseanet assets URIs.

Cool URIs don’t change : the most important thing is that the URI should never change.

Phraseanet offers many tools for server side video/image operations. If the media is updated (color correction, rotation, crop….), image cache should be invalidated.
For that reason we choose the cache HTTP Validation model (https://tools.ietf.org/html/rfc2616#section-13.3) :

This model provides a validation per request and you are sure to get an up-to-date content every time you update the DOM.

As of this model, every image is retrieved within a response that includes two HTTP headers : ETag and LastModified.

Etag: "6df6abcb24976f126a97d58b4ef44ce6"
Last-Modified: Fri, 22 Jun 2012 17:50:05 GMT

The browser will use them for the next image load. Its request to the server will include two headers :

If-Modified-Since Fri, 22 Jun 2012 17:50:05 GMT
If-None-Match: "6df6abcb24976f126a97d58b4ef44ce6"

If the image has not changed since the last load, the server should return a 304 response with an empty body. Otherwise, the image is returned with the new ETag and LastModified values.

After implementing it I realized that in a Single Page Application case, only Chrome was working as expected.

In this context, other browser do not validate image cache. It is validated only at a first document inclusion.
I mean, that if you already have displayed the image in the Document, then image cache will not be validated at next inclusion but only after next document load in Safari, Firefox or Opera.

I submitted three tickets :

Firefox guys are the first one that have answered, and as you will see, to be strict standard compliant is not that easy today.

This article will end with a link to this nice Symfony Documentation about HTTP caching.

Asynchronous template load within jQuery UI autocomplete widget

Posted on by
Reply

This week I was playing with jQuery UI Autocomplete widget. I usually use Mustache.js as a template engine for rendering HTML, and I load templates on-demand.

My problem was that the widget was not ready for asynchronous load.

I use this code to load the Mustache template and render it in the Autocomplete widget :


    var MustacheRender = function(TemplateName, datas, callback) {
      $.ajax({
        type: "GET",
        url: "/MustacheLoader/",
        dataType: 'html',
        data: {
          template: TemplateName
        },
        success: function(template){
          return callback(Mustache.render(template, datas));
        }
      });
    };

    $('input').autocomplete()
      .data( "autocomplete" )._renderItem = function( ul, item ) {

        var LoadAndRender = function(datas){
          $(datas).data( "item.autocomplete", item ).appendTo( ul );
        };

        MustacheRender('List-Item', item, LoadAndRender);
        return;
      };

In this case, everything is visually ok, but it does not work as expected. If you try it, you will find out that mouse* or key-based events are not triggered.
This is because the item list has not been “refreshed”.

As a workaround, you can manually refresh the list by adding this to the “LoadAndRender” callback:

$('input').data( "autocomplete" ).menu.refresh();

It now perfectly works !

Multilingual documentation with sphinx

Posted on by
Reply

These last days, we were looking for a documentation system for the upcoming Phraseanet release. We were used to build the documentation on an old and fixed Microsoft-like documentation system (sometimes called Windows-Help).

We needed a tool that can handle multilingual documentation, easy to use, flexible and easy to transform (HTML, PDF, ….).

We tried many wikis. It was not flexible enough. Export methods were particularly very weak. Until we had a look at ReStructured Text. We now love it !

It is as easy as a wiki to start. The multilingual difficult was solved with multiple documentation roots – one per language. The build is managed by one Makefile wrote with the help of CakePHP ( thank you http://mark-story.com/posts/view/creating-multi-language-documentation-with-sphinx )

Here’s the result : https://github.com/alchemy-fr/Phraseanet-Docs, you can see the html build at http://docs.phraseanet.com

More resources to start your project : http://sphinx.pocoo.org/tutorial.html

Symony2 Response Object

Posted on by
2

I’ve been playing a lot with the Silex Microframework these last weeks.
I had to build an API for an existing code.
The result is amazing, Silex let us build our API, write full coverage unitTests in no more than a week.
I will write a dedicated post about Silex later, the subject is now the Response Object of the Symfony2 HttpFoundation Component :

I found a disturbing behaviour in this object :

  • If you do not set a charset to this object => HTTP Content-Type header is marqued  ‘UTF8′ for “text/*” content-type (defaut content-type is text/html).
  • or If you set a charset by defining it yourself in the header => HTTP Content-Type header is marqued as your charset (good)

But, in both case, the Response::getCharset method will return null.

This is disturbing for the Unit Tests.
Actually, the WebTestCase component provided by Silex get the Response object, when testing a route. So when I check the charset, the test fails.

<?php
require_once dirname(__FILE__) . "/Silex/autoload.php";

use SymfonyComponentHttpFoundationResponse;

$app = new SilexApplication();

/**
* Good HTTP response / Good Behavior of Response::getCharset
*/
$app->get('/testpass/', function() use ($app)
  {
    $response = new Response('Hello world');
    $response->headers->set('Content-Type', 'text/plain');
    $response->setCharset('UTF-8');
    return $response;
  }
);

/**
* Good HTTP response / Bad Behavior of Response::getCharset
*/
$app->get('/testfail/', function() use ($app)
  {
    return new Response(
                      'Hello world'
                      , 200
                      , array('Content-Type' => 'text/plain; charset=UTF-8')
                );
  }
);

I’ll try to enhance this behaviour in the next few days.