Tag Archives: PHP

PHPConsistent : a new tool to verify your calls and documentation quality

Back in 2009 and 2010 I wrote about a PHPUnit patch I wrote to automatically verify parameter types in function calls. The feature never made it into PHPUnit and honestly it didn’t really fit into the feature set either. Although I still plan on releasing it as a PHPUnit extension that you can easily load, I’ve since been using it outside of PHPUnit, not just on tests, but on any PHP code.

Introducing PHPConsistent

PHPConsistent will verify your code using both dynamic and static analysis.

The goal is to improve code quality of your code and the libraries you use by :

  • Verifying your code is making calls using the right parameters and parameter types
  • Verifying if the in-line documentation (docblock) of the called functions/methods is accurate

It will compare :

  • Parameter types specified in the docblock <-> types of parameters passed upon calling the function/method
  • Number of parameters specified in the docblock <-> number of parameters actually present in the function/method definition
  • Names of parameters specified in the docblock <-> names of parameters actually present in the function/method definition

Sample output

Invalid type calling SomeClass->GiveMeAnArray : parameter 3 ($somearray) should be of type array but got boolean instead : library/App.php (line 5)
Parameter names in function definition and docblock don't match when calling JustAnotherFunction : parameter 2 ($inputFilename) should be called $inputFile according to docblock : application/Bootstrap.php (line 214)
Parameter count in function definition and docblock don't match when calling OneMoreFunction : function has 6 but should be 5 according to docblock : application/Bootstrap.php (line 215)

Performance

Keep in mind that PHPConsistent relies on Xdebug’s trace functionality, making it quite slow. It also needs to analyze the output of that trace, making it even slower. So it’s definitely not something you want to run on a production environment !

Want to know more ?

Check out the PHPConsistent Github page

PHPCompatibility update

Update (Nov 3, 2013) : changed the parameters to reflect the new changes in PHP_CodeSniffer

Just a quick post about the updates made to PHPCompatibility.

I did a talk on PHP 5.4/5.5 at Zendcon, not just about what’s new and why you should upgrade, but also about the way PHPCompatibility can ease the pain. I received very good feedback, as well as very interesting questions. As a result, I added a feature I’ve wanted to add for a long time : the option to specify which PHP version to check for.

Here’s the caveat : it requires a change to PHP_CodeSniffer. I’m hoping that change will be accepted soon. From that point onwards, if you have the latest PHP_CodeSniffer (you might want to get it from Github).

So then you will be able to do :

phpcs --standard=PHPCompatibility --runtime-set testVersion 5.4 <path-of-your-php-files>

You can also test for backwards compatibility of your code by specifying versions like 5.2, which causes it to check whether you’re using functions or keywords not known to this older version. Quite nice if you’re coding on PHP 5.5, but your system engineer tells you half way through the project the project will have to be deployed on 5.2.

Important note : if you want to test backwards compatibility, it’s best to run on the latest PHP version available (5.5 right now), for the simple reason that if you have, for example, the keyword ‘yield’ in your code, PHP_CodeSniffer can only recognize it when you’re running PHP 5.5. It can only tell you it’s not available in previous versions if it actually knows what it is.

Expect more compatibility tools soon, as I have some more ideas, such as how to automatically figure out whether and where you’re still relying on register_globals and magic quotes 😉

Conferences, development, ideas, …

I just returned from PHPCon Poland, a very nice conference taking place in the middle of beautiful scenery. Although there were only 5 talks in English (3 of which were given by Belgians – Thijs Feryn, Michelangelo Van Dam and myself), the conference was definitely interesting and fun.
I gave a relatively new talk, that I first presented as a 20-min ‘lightning talk’ at ZendCon Uncon 2011. It’s evolved into a 45-min talk (ok, it was only 38-min so I can add some content next time) titled ‘Remove PHP calls and scale your site like crazy’ in which I explain more about the Nginx extension we’re building to improve the performance and scalability of sites with user-specific content. I received some pretty good feedback on Joind.in and lots of people approached me afterwards, wanting to learn even more about it.

I also submitted 6 talks for Confoo, which is supposed to be one of the best web development conferences out there. Not sure if I’ll make it, but if you want to help out, feel free to vote for me. Registration is required to vote, but takes only a minute.

Lots of ideas buzzing in my head, but sadly not enough time to work on them. The Nginx thing is cool, but it’s what I call ‘phase 1’… which means there’s a ‘phase 2’ (in fact, there’s even a ‘phase 3’). But since that requires even more time, it’ll probably be for when phase 1 has been completed.

In the meantime, we’re on the hunt for people looking for a new challenge. If you’re looking to be seriously challenged, check out our jobs page and feel free to tweet/mail/call me 😉

PHP 5.4 compatibility checks using PHP_CodeSniffer

Update (27 Nov) : Support for PHP 5.5alpha is included.

For those of you who are new to this concept, check my blog post from a while ago. It will explain the basic concept of using PHP_CodeSniffer to automate compatibility checks. But don’t use the download links, because they point to the old (PHP 5.3) version !

What’s new ?

Quite a few things have changed in this new release :

  • There’s no version specific release anymore. The previous codesniffer standard was called PHP53Compatibility, but it seemed quite stupid to make a new standard for every PHP version out there, especially since that would keep certain people from upgrading to the latest major PHP version. So the new PHPCompatibility standard works for 5.0 – 5.4
  • But since some people simply can’t upgrade to the latest version, I added version information to all the checks. For example : the deprecated  function checker will now tell you that session_register() is deprecated since PHP 5.3 and removed since PHP 5.4 – if you’re running 5.2 and want to move to 5.3, at least you know right away that you’ll have to fix that problem, because otherwise you can’t ever upgrade to 5.4. This version check is available on deprecated/removed function, deprecated/removed php.ini directives and deprecated/removed extensions.
  • Default timezone check has been added : since PHP 5.4, you need to have a default timezone set or PHP will complain. This is ofcourse only useful if you run the tests on a system with identical settings as your production environment.
  • A check for the removed functionality on break and continue was added. (Using a variable or function call as a parameter on break and continue is no longer allowed.)
  • 2 algorithms were removed in the hash extension, so there’s a check for that as well

Where to get it

2 options :

  • Using git : run this in your PHP_CodeSniffer/Standards directory :
    ~ > git clone git://github.com/wimg/PHPCompat_CodeSniffer.git PHPCompatibility
  • Downloading a zip : download here and unzip the file in PHP_CodeSniffer/Standards/PHPCompatibility

How to run it

Start PHP_CodeSniffer like this :
phpcs --standard=PHPCompatibility

Enjoy !

As always, any feedback (or patches on Github) welcome !

Using PHPUnit to verify parameter types (revisited)

(This is an update on a blog post I wrote last year about parameter type checking)

PHP is dynamically typed

PHP is a dynamically typed language. What this means is that it allows you to do things like :

$a = 5;
$a = 'test';
$a = false;

The reason this works, is because PHP enforces type rules during execution, not at compile-time.

In many other languages this is impossible, since you need to define a type for the variable at compile-time. Languages such as Java, C, C++, C# and VB.Net are good examples of statically typed languages.

Problems with dynamic typing

Although dynamic typing is considered to be one of PHP’s strong suites, it does pose some problems. Let me illustrate with an example :

Suppose we have a piece of code that processes the amount of money each employee must be paid. Employees can file expense notes that are paid back in cash or when their monthly wages are paid. Our code will make the calculation for the pay check.

The data for our 4 employees is located in a CSV-file, made available from an external source :

employeeId, firstname, lastname, wage, expenses, processexpenses
1, Claire, Clarckson, 2000, 212, 0
2, Tom, Whitney, 1910, 111, 0
4, Jules, verne, 1932, 98, 1
5, Gregory, Jameson, 2131, 241, 0

If the last field is true, the expenses must be added to wage amount on the paycheck. So our code might look like this (don’t pay attention to code quality, it’s an example of ‘the average piece of code you will find’) :

class Wages
{
/**
* Process the wages
* @return boolean
*/
public function processWages()
{
$handle = fopen('some-file.csv', 'r');
while (($data = fgetcsv($handle, ',')) !== false) {
if (is_numeric($data[0])) {
$result = $this->processLine($data[2], $data[3], $data[4]);
$this->sendPaycheck($data[0], $result);
}
}
return true;
}

/**
* Calculate wages based on processExpenses parameter
*
* @param float $wage
* @param float $expenses
* @param boolean $processExpenses
* @return float
*/
private function processLine($wage, $expenses, $processExpenses)
{
if ($processExpenses) {
return $wage + $expenses;
} else {
return $wage;
}
}

/**
* Pay the employee
*
* @param int $id
* @param float $amount
*/
private function sendPaycheck($id, $amount)
{
echo 'Paycheck for id ' . $id . ' for the amount of : ' . $amount ."\n";
}
}

Everything works fine, until the external source decides (probably unknowingly) to modify the data format to :

employeeId, firstname, lastname, wage, expenses, processexpenses
1, Claire, Clarckson, 2000, 212, false
2, Tom, Whitney, 1910, 111, false
4, Jules, verne, 1932, 98, true
5, Gregory, Jameson, 2131, 241, false

When we run the code, everyone will be paid their expenses, even those that have ‘false’ in the last field. The reason ? The last field of each line might look like a boolean, but is in fact a string. The “false” is read as a string and is boolean true.

You might say that we didn’t follow best coding practices in our :

if ($processExpenses) {

which should have been

if ($processExpenses === true) {

but that would only have reversed the effect : nobody would have been paid.

Similar non-boolean situations cause the same problem. There’s a huge list of problems that might be caused by passing incorrect types to a function.

Granted, we should have put a type check in place, but as I said this was the average type of code you will find. And it’s exactly this type of code I wanted to use for this demo.

So what’s the solution ?

Since we don’t want to give up on our dynamic typing, we need a way to verify that parameters being passed to a function/method are of the type that we intend them to be. That way, anyone who wishes to use our function/method will be forced to pass the right parameter.

One solution would be to use type safe objects like the ones described by Sebastian Bergman (author of PHPUnit) in his Type-Safe Objects in PHP presentation. However, this is unusable for existing projects as it requires a massive rewrite. Furthermore, as Sebastian indicates, it poses a lot of new problems. And finally, it slows things down quite a bit, since it uses reflection to verify types during execution…

Another solution would be to have type hinting for all types, including scalar types, in PHP. Although proposed and agreed to by many, the current concensus (as of today at least) is to not include it PHP 5.4. It might end up in a branch for future use or might end up as a PHP extension down the line, but for now it seems to be off the table for PHP 5.4.

So should or shouldn’t we check the type of a parameter before using it ?

Another big dillema : should you check each parameter’s type in each single function at runtime using is_int, is_bool, etc. ? Some would say it’s the safest way and the only way to be absolutely sure.

I believe there’s a different and better approach : if you can integrate type checking in your unit testing and have a high code coverage percentage, there’s no need to explicitely check the type during runtime, except when handling external data.

So how do we make sure types are checked ?

What system is better suited for the job than the most popular testing PHP framework, PHPUnit ?

Since PHPUnit run can be repeated over and over again and introducing additional checks will not cause performance issues on the actual production environment, this is a good place to add these checks.

Upon execution of each PHPUnit Testcase, this patch will verify parameters for each of the called functions/methods. You can define the depth of calls using a PHPUnit command line parameter (the default is 2, which means functions called from the testcase itself and functions called from those functions).

How does the system know what types are expected if we don’t have type hints ?

The system presumes that if you’re using PHPUnit, you clearly know proper development methods. This also means you’ll be using docblocks to comment your functions.

So, since there are no type hints to rely on, it will instead rely on the types you specify in the docblock

It will analyze the docblock of each function/method and compare each parameter type with the expected parameter type. If it finds an inconsistency, it will produce a PHPUnit warning.

So does it support…

– Classes : yes

– Interfaces / Abstract classes / parent classes : yes, in fact if you specify an interface or abstract/parent class in the docblock and pass a class implementing/extending them, it will detect it as a valid type

– Multiple types definitions in the docblock : yes – separate them by a pipe (|)

– Return value types : yes – needs Xdebug patch, see below

Is it perfect ?

Nothing is… there’s a few problems at this point :

– If you want to analyze return values, you’ll need a patch for Xdebug I wrote last week. You can download that patch here : XDebug bug #416 patch

– It still needs a bit of tuning… it’s a work in progress !

The MySQL problem : something most people don’t know

Data from any external source might cause problems. MySQL is the best example : any data being returned from MySQL using the non-binary protocol is a string, even if the column is defined as int, decimal, bool (tinyint), …

MySQL’s protocol returns all data as a string and the PHP mysql and mysqli extensions don’t convert it into the expected datatype. The result is that any data from MySQL will be passed as a string, which can cause havoc when doing type checks. The only exception is when using mysqlnd and prepared statements (see the second example of Scalar type hints in PHP trunk).

There are 3 solutions to this problem :

  • Cast everything : not really fun, since you’ll need to change all your code. It might also be bad for performance, although it does set the types right ofcourse. Might be the best option for new code.
  • Use Propel or some other database layer, which does the casting for you… same performance problem ofcourse.
  • Wait until someone makes changes to the way MySQL and PHP communicate…

In the meantime, if you want to use the type checking, but you have some problems with MySQL, you can use a docblock tag to disable type checking for some functions : @phpunit-no-type-check

How to run it

After applying the patch to PHPUnit (and Xdebug if you want return type checking), run it like this :

php [path to phpunit.php] --check-param-types [TestCase.php|TestSuite.php]

Optional parameter :

–check-param-type-depth= sets the depth to which it needs to check parameter types. Your test is depth 0, any called function within your test is 1, etc. – default is 2 although 3 might be handy too

The output

This is the kind of output you can expect :

2) ATest::testMultiply

Invalid type calling A->multiply : parameter 2 ($factor2) should be of type int but got string(1) instead in C:\development\Test\ATest.php:42

Note that if you use the ‘–check-param-type-depth’ parameter and set it to a high number, you might see errors about libraries you use. Ofcourse, that might be the right moment to notify the library author (or contribute a fix yourself !)

Advantages

  • No need to use is_int, is_bool, etc. in a function that was type-checked
  • Consistent use of types
  • Developers will learn to focus not just on the content of a variable, but on the type as well. In time, it will become second nature to use the correct types from the start. Code hinting in most IDEs should in fact already help out with this, but now there’s a way to verify this too.
  • Verification of each function/method call, not just in terms of functionality (PHPUnit’s job), but also in terms of data types
  • Forces developers to keep the docblock up-to-date (!)

Using type checking basically brings the best of the dynamically and statically typed worlds together : you still have the flexibility of dynamic typing, but assurance that functions are called with the parameter types they were designed to be called with (as well as return the correct types). It’s the perfect middle-of-the-road approach for teams with a mix of ‘strict’ and ‘not-so-strict’ developers.

Where to get it

The PHPUnit modification can be downloaded through a Github-fork : http://github.com/wimg/phpunit

Possible extension

There’s plenty of things that could be added.

One of them is “non-strict” option, which ignores type conversion between types as listed on http://wiki.php.net/rfc/typecheckingstrictandweak (option 2 section)

Feedback

As always, feedback is much appreciated, as a comment in this blog or via e-mail.