Zend Certified Engineer!
Deploying PHP applications with Phing (PHPNW11)
Below are the slides of the Phing talk I gave during the PHPNW11 conference.
Building and deploying PHP applications with Phing (PHPUK2012)
Below are the slides of my Phing talk I gave at the PHP UK Conference 2012.
Also, some of the examples discussed during the talk can be found here.
Phing development update
Starting today, I’ll post regular updates on the development of Phing (which is, as you might know, a PHP build tool based on / inspired by Apache Ant).
For the past three years I’ve been the lead developer on this open source project, and thus responsible for deciding the direction of the project, integrating the various contributions, publishing releases, etc. Should you be interested, Lorna Jane Mitchell published an excellent post detailing some of the challenges faced by open source project leads.
The last few months have been especially interesting, as I’ve moved the Phing source code to GitHub. This has greatly improved the number of contributions and in general speeded up the development of Phing.
Let’s discuss some of the interesting commits that were made during the past two months!
Try/Catch task
A feature that has been requested before (and has been on my personal wishlist) is the ability to run task(s) whenever a task fails (very similar to PHP’s try/catch statement). During my talk at the PHPUK2012 conference, one of the attendees asked when this feature could be expected. So, while waiting for my flight home I decided to get busy and add the trycatch task.
<trycatch property="prop.testTryCatchFinally"> <try> <fail>Tada!</fail> </try> <catch> <echo>In <catch>.</echo> </catch> <finally> <echo>In <finally>.</echo> </finally> </trycatch>
Parallel task
After a very interesting talk by Jeroen Keppens (@jkeppens) during this year’s PHP Benelux conference, Mike van Riel (@mvriel), the author of DocBlox, and me set out to enable some form of parallel processing in both Phing and Docblox. Mike authored a generic library, which I used in the new parallel task. This task is still very experimental (and it only works on Linux due to some OS limitations), but it’s definitely a nice example of how the community works and inspires!
<parallel threadCount="100"> <echo>Job 1</echo> <echo>Job 2</echo> <echo>Job 3</echo> <echo>Job 4</echo> <echo>Job 5</echo> <echo>Job 6</echo> <echo>Job 7</echo> <echo>Job 8</echo> <echo>Job 9</echo> <echo>Job 10</echo> <echo>Job 11</echo> <echo>Job 12</echo> <echo>Job 13</echo> <echo>Job 14</echo> <echo>Job 15</echo> <echo>Job 16</echo> </parallel>
Liquibase
Together with Stephan Hochdörfer (@shochdoerfer) I’ve been working on integrating his Liquibase (a Java database refactoring / change management application) tasks into the Phing core. The liquibase tasks will complement the dbdeploy task and assist the user with database changes / rollbacks.
ApiGen
ApiGen is a Nette framework based documentation generation tool for PHP 5.3. Phing now supports this tool through the new apigen task, contributed by Jaroslav Hanslík.
RNG schema updates
For a while now, Phing ships with a RNG schema to validate your build files. However, this schema was incomplete and did not contain some of the more recent tasks and changes. Johan Persson has done some invaluable work to get the RNG schema up to date.
Various (bug)fixes, the 2.4.10 release
Steady progress has been made towards the 2.4.10 version (which is scheduled to be released early April). Below is an overview of the tickets that have been closed so far:
Besides those commits, a good amount of other (small) fixes and contributions will make it into 2.4.10. A complete list of commits can be found here. Keep those pull requests coming!
Joind.in nominated for an Impact Award
The community site joind.in is the definitive community event tracker, and it’s entirely open source! Today I learned that the site has been nominated for the 2012 Impact Awards. As a conference visitor and speaker, I visit and use this site regularly (you can find my speaker profile here) and am really excited for this nomination!
Please vote here!
My Ideas For March
Last year, Chris Shiflett started the Ideas for March movement, an effort to revitalize the interesting conversations that happen(ed) on blogs. This year, several people are refreshing that effort.
This is my pledge to write (blog) more about what I see and do, the projects that I work on, the tools that I use daily, the people I meet, the events I visit – the things that inspire me. I will be blogging about all things Phing, hopefully post a blog or two about Gearman, SOAP, ZF, security and numerous other subjects that have been stuck in my grey matter, but never managed to make it into a blog post.
Plus, and I’m very excited about this, my good friend Joeri is busy working on a redesign of this site, which will hopefully make all that new content more appealing and easy to read!
Phing development update 01/04/2012
Roughly a month ago I posted the first in, what should become, a series of development updates concerning Phing. This second blog post details the most important updates during the past four weeks as Phing is moving steadily towards another release (2.4.10 is due out next week).
User Guide starts move to DocBook format
The biggest commit last month was without a doubt the merge of Johan Persson’s docbook5 branch, which adds a DocBook 5 version of the entire Phing User Guide, accompanied by its own rendering infrastructure. This feature has been on my personal wishlist for a long time, and it’s finally around the corner! The new docs will probably not make it into 2.4.10, but, at some point in the coming weeks, the ‘live’ user guide (on the Phing website) will be switched to the new format. The 2.4.11 packages will also contain the new docs.
Liquibase docs
After succesfully merging the Liquibase tasks last month, Stephan Hochdörfer completed the addition by submitting a pull request containing the necessary documentation!
PhpDocumentor
You’ve probably heard about this, the popular documentation tools phpDocumentor and DocBlox have merged! Phing supports the new phpDocumentor releases by adding a phpdoc2 task.
PHP 5.4 compatibility
Phing itself seems to work fine with the recently released PHP 5.4, however, the unit tests experienced some problems – these have been corrected to make sure the (unstable) snapshot builds are pushed out regularly.
PHAR package
Starting with the upcoming 2.4.10 release, each new version of Phing will also be available as a PHAR package. Initially, this will just contain Phing itself (similar to the PEAR package). After that, the package will be expanded with a few popular dependencies (depending on feedback).
Various (bug)fixes, the 2.4.10 release
Last but not least, here’s a list of tickets that have been closed since the last update.
A new year…
Remember how I promised to blog more?
I promise I’ll improve this year Meanwhile, visit this brilliant site.
Phing development update
The last Phing development update was almost a year ago, so I guess it’s about time for a discussion of what’s new in Phing world.
2.5.0 and Semantic Versioning
The most important news in this update is the brand new 2.5.0 release (pushed out yesterday). We bumped the minor version number because we now (try to) follow the Semantic Versioning spec, and this release introduces new functionality. Our changed roadmap reflects this as well.
This version closes the following issues:
Other goodies
The past few months Phing has seen a few interesting additions:
- Composer packager support (we’re listed on Packagist as well)
- phploc task
- apply task
- improved phpdoc(2) support
- lots of documentation, bug and other fixes
GitHub
The move to GitHub in early 2012 has been very successful in speeding up Phing’s development. We can definitely still use your help though, so fork our project and submit a pull request. Thanks!
Interesting links
Some interesting links I’ve stumbled on the past few weeks, in no particular order:
Phing development update
It’s about time I wrote another Phing development update post, as the previous one was written 6 months ago!
So, what have we been up to ?
Release 2.5.1
Unfortunately, because of other commitments, the 2.5.1 release has been delayed a little and should hopefully be pushed out later this month.
Here are some of the issues that have already been fixed for this release:
DocBook manual
A number of awesome pulls by johan162 updated our docbook manual (which has been in the works for some time) to a viable alternative. Next on the list are a few infrastructure issues and styling changes, after that the current manual will be ready to be replaced by our shiny new docbook version
Details:
- https://github.com/phingofficial/phing/pull/203
- https://github.com/phingofficial/phing/pull/212
- https://github.com/phingofficial/phing/pull/213
- https://github.com/phingofficial/phing/pull/214
- https://github.com/phingofficial/phing/pull/217
Other changes
For more in-depth information, see the list of recently closed pull requests and the list of commits.
Using conditional build steps to speed up your Jenkins PHP builds
At my client Spotney, we have a pretty solid (and common) build infrastructure for our PHP projects; SVN commits are checked out by Jenkins, tests are run by PHPUnit, Sonar runs static code analysis, and artifacts are built and deployed to a staging environment by Phing. However, some of the code relies pretty heavily on (complex) db queries, adding the need for DbUnit style tests. The nature and quantity of the tests, combined with a slow VM (possibly related to this Xdebug issue) meant that our buildtimes were becoming prohibitively long.
An interesting post by @pascaldevink triggered a conversation and sparked an idea. I started working on our build setup, eventually resulting in a 60-70% decrease of our build times!
Here’s how I did it.
Starting point
Let’s assume we have a fairly standard Jenkins job. The job checks out an SVN repository, and periodically scans that repository for changes, triggering a new build if any are detected.
Each build of the job performs three steps:
- Run phpunit
- Run phing (target “build”)
- Invoke Sonar (using the Jenkins Sonar plugin - this plugin also allows invoking Sonar as a post-build action, but that option requires Maven)
After the build steps, the job publishes the test and code coverage reports, and archives the generated artifacts.
Disabling code coverage and Sonar for regular builds
Two of the most obvious optimizations (also suggested by Pascal) are disabling code coverage on all tests and disabling Sonar runs during regular Jenkins builds. We define regular as either manually started by a user, or by an SCM trigger.
Disabling code coverage generation in PHPUnit is easy, simply remove the “coverage-xxx” elements from the logging section of your phpunit.xml configuration file (see this section of the PHPUnit manual). Disabling Sonar is trivial too, just remove the last build step from the job.
However, this is not an ideal solution: we do want to generate code coverage information and run Sonar at some point, such as during nightly builds, preferably without cloning our existing job. This means that we’ll need to skip code coverage and Sonar invocations on all but the scheduled (nightly) builds.
The Sonar plugin supports excluding SCM triggered builds (“Skip if triggered by SCM Changes”), but that only works if you use the post-build action. Additionally, we need to be able to change the PHPUnit configuration – one file to enable code coverage generation, one file to disable it.
Conditional build steps
The Conditional BuildStep plugin wraps one or more build steps in a conditional execution block. One of the possible conditions is the build cause, i.e. was the build triggered by a user, an upstream project, a timer, an SCM change, etc. etc.
First we define the steps that should be taken for each nightly build of our job. These steps should only be executed when the build is trigger by a timer.
We add a “Conditional steps (multiple)” build step, setting the condition to “Build Cause” and the Build Cause to “TimerTrigger”.
Then we add our three (original) build steps:
As stated before, regular builds are those that are triggered by a user or an SCM change.
We again add a “Conditional steps (multiple)” build step. The condition for this step is a little more interesting, as seen below. We combine two Build Cause conditions (one set to “UserCause”, the other to “SCMTrigger”) using the Or condition.
We then add two build steps: the first will run PHPUnit without code coverage (note that we are specifying a different configuration file here), the second one will run Phing.
Note that in the above build steps we’re invoking Phing from the shell instead of using the Phing plugin. Unfortunately this plugin is currently not supported as a conditional build step (probably because of this JIRA issue).
Build schedule
As a final step we need to update our build schedule.
This will ensure our job runs somewhere after midnight (between 12:00 AM and 2:59 AM to be precise).
The end result:
- A nightly scheduled build, with all the bells and whistles enabled
- User and SCM triggered builds run (a lot) faster
Please let me know if you think this post is useful, or if you have any other Jenkins/PHP optimization tips!
Dutch Web Alliance, leaders in web technology
I’m proud to announce that I’ve joined forces with a group of very talented web developers in the Netherlands and Belgium: the Dutch Web Alliance.
More information is available on our website, the full (dutch) press release is posted below:
APELDOORN: Freelance web developers bundelen krachten in de Dutch Web Alliance.
De Dutch Web Alliance (DWA) is een vereniging voor en door freelance
web developers uit Nederland en Belgie. Onze missie is om via kennisdeling, samenwerking en het combineren van resources, de positie van de freelancer en de kwaliteit van web professionals te verbeteren. Een lid van de DWA staat voor kennis en kunde op alle vlakken in web-ontwikkeling en kan zich als zodanig onderscheiden worden door klanten en opdrachtgevers.De DWA richt zich niet op kwantiteit, maar op kwaliteit. Zo zijn het de leden onderling die bepalen of een nieuw lid ook daadwerkelijk kan toetreden tot de vereniging. Binnen de vereniging houden leden elkaar actief op de hoogte van alle laatste ontwikkelingen, bieden ze hulp en dienen ze als vraagbaak. Maar DWA leden springen ook graag bij andere leden in situaties zoals ziekte, deadlines, specifieke kennis. Zo krijgen afnemers van de DWA de flexibiliteit van freelancers, met de voordelen van full-service internet bureaus.
De DWA is meer dan alleen een keurmerk en kwaliteitswaarborg. Onze leden zijn voor het grootste deel actief in open source communities zoals PHP, dev/ops en system administration. Zij zijn dan ook vaak terug te vinden op nationale en internationale conferenties als spreker waarbij ze als experts en “leaders of the field”, andere ontwikkelaars helpen met nieuwe technieken, best practices en algemene kennis die is opgedaan uit de vele complexe projecten waar ze dagelijks aan werken. De DWA draagt hier nog een extra steentje aan bij door het organiseren van workshops en meetups voor niet alleen ontwikkelaars, maar ook leidinggevenden, project managers en CTO’s/CIO’s. Kortom: onze vereniging zet zich in voor een beter, professioneler en efficiënter klimaat binnen de web-ontwikkeling.
Docker & PHP: beyond virtual machines
(note: this post was first published on the blog of the Dutch Web Alliance)
Docker is currently one of the hottest technologies around, because it solves a very specific problem: the ability to easily package and deploy a (self contained) application, without the overhead of traditional virtualization solutions.
In this post you’ll learn how to build, run and host Docker containers, integrate with other containers, and see how Vagrant interacts with Docker.
Background
The Linux kernel contains a number of containment features that enable resource (CPU, network, memory, etc.) & process (user id’s, file systems, process trees) isolation on the same host, without the need for a virtual machine: cgroups and namespaces.
This operating system-level virtualization imposes far less overhead. User space instances (also called containers) run within the host OS, so a hypervisor and guest OS are not needed. This results in lightweight images and incredibly fast start/boot times: think seconds, not minutes. Containers are like vm’s, without the associated weight.
Docker is a toolkit built around those containment features. Before version 0.9, Docker relied on LXC (LinuX Containers): a (userspace) interface to the kernel containment features. As of version 0.9, Docker has its own interface layer called libcontainer.
Installing Docker
Installing Docker is easy if you are using Ubuntu. To install the official package (may not be the absolute latest version):
$ sudo apt-get update $ sudo apt-get install docker.io $ sudo ln -sf /usr/bin/docker.io /usr/local/bin/docker $ sudo sed -i '$acomplete -F _docker docker' /etc/bash_completion.d/docker.io $ source /etc/bash_completion.d/docker.io
Should you really want the latest version:
$ curl -s https://get.docker.io/ubuntu | sudo sh
Note: because Docker relies on Linux-specific kernel features, it is not natively supported on OSX or Windows.
There are some workarounds: use Vagrant (see below), or refer to the Docker installation instructions ( https://docs.docker.com/installation/windows/ and https://docs.docker.com/installation/mac/) for details.
Creating and starting a container
Let’s start with a small PHP app, which we will package and deploy as a Docker container. The app will expose a simple socket server on a port (1337), and output a string to connections on that port. To build the app we’ll be using React, a library that adds event-driven, non-blocking IO to PHP.
First, install composer (http://getcomposer.org), then run the following command:
$ composer require react/react=0.4.x
Then save the following code as “app.php”:
<?php require 'vendor/autoload.php'; $loop = React\EventLoop\Factory::create(); $socket = new React\Socket\Server($loop); $http = new React\Http\Server($socket, $loop); $app = function ($request, $response) { $response->writeHead(200, array('Content-Type' => 'text/plain')); $response->end("Hello World\n"); }; $http->on('request', $app); echo "Server running at http://0.0.0.0:1337\n"; $socket->listen(1337, "0.0.0.0"); $loop->run();
If we then run the following command:
$ php app.php
we should see something like this:
Server running at http://0.0.0.0:1337
Open your browser, point it to http://localhost:1337/, and you should see something like:
Now, to get our app to run inside a Docker container, we first need to build an image. The instructions that define how to build the image are saved in a Dockerfile, in the form of commands:
FROM jolicode/php55 ADD ./ /var/apps/reacttest CMD cd /var/apps/reacttest && php app.php EXPOSE 1337
In this Dockerfile we’ve used the following commands:
- FROM: an existing image that our image should be based on
- ADD: a tar file or existing directory pointing to our app
- CMD: command to run when starting the container
- EXPOSE: ports that we want accessible outside of the container
The full list of commands is available .
Now that our Dockerfile is finished, let’s build the image. Run the following command:
docker build -t "our_app_image" .
This builds a new image using the Dockerfile in the current directory, and will tag
the resulting image with the string “our_app_image”. The output of the command should look like this:
Sending build context to Docker daemon 1.633 MB Sending build context to Docker daemon Step 0 : FROM jolicode/php55 ---> 13f8ff6325db Step 1 : ADD ./ /var/apps/reacttest ---> 09b79e178ff3 Removing intermediate container 20296bc15647 Step 2 : CMD cd /var/apps/reacttest && php app.php ---> Running in 8d907cb22524 ---> 66feac5b7e2c Removing intermediate container 8d907cb22524 Step 3 : EXPOSE 1337 ---> Running in 42b14e4d20d3 ---> ad72a42f978a Removing intermediate container 42b14e4d20d3 Successfully built ad72a42f978a
The next step is creating and running a new container that uses our freshly baked image.
docker run -d --name "our_app_container" -p 1337:1337 our_app_image && docker ps
The container will be named “our_app_container”, and will run in the background (the ‘-d’ option). Docker will also proxy port 1337 on the host to port 1337 in the container. After starting the container, ‘docker ps’ shows the list of running containers and their attributes, something like:
5c35b78eede36eb34ca7498b202979200df56dce6343cc5f0d8242f370bba64f CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 5c35b78eede3 our_app_image:latest /bin/sh -c 'cd /var/ Less than a second ago Up Less than a second 0.0.0.0:1337->1337/tcp our_app_container
To verify that the app is running and available, we can use ‘docker logs app’ to retrieve the stdin/stderr streams of the running container.
Dependencies: linking containers
Let’s extend our app with an external dependency. The requirement: increase a counter with every request to port 1337. The counter will live inside a Redis instance.
First, add predis-async to the list of composer dependencies:
$ composer require predis/predis-async=dev-master
Then, insert the following lines in app.php:
$client = new Predis\Async\Client('tcp://redis:6379', $loop); $app = function ($request, $response) use ($client) { $client->incr("app:requests"); $response->writeHead(200, array('Content-Type' => 'text/plain')); $response->end("Hello World\n"); };
Now, how to add a Redis server to our app? We can connect directly to a specific hostname, or assume that whoever works on our app has Redis installed and running. Both are somewhat fragile, and make setting up a development environment more difficult. There is a third option: installing Redis directly to the Docker image of our app. However, this would polute the image and make it less flexible.
Enter Fig, a tool to quickly build and start (isolated) development environments. Fig requires a single configuration file (default: fig.yml):
app: build: . links: - redis redis: image: redis
The above file defines two containers: app (which contains our php application), and redis. The first container depends on the latter through the “links” section. Fig will make sure that, within the app container, “redis” resolves and points to that container.
Save the file, and run:
$ fig -p app up
This should produce the following output:
Creating app_redis_1... Creating app_app_1... Attaching to app_redis_1, app_app_1 redis_1 | [1] 09 Nov 10:19:23.272 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf redis_1 | _._ redis_1 | _.-``__ ''-._ redis_1 | _.-`` `. `_. ''-._ Redis 2.8.17 (00000000/0) 64 bit redis_1 | .-`` .-```. ```\/ _.,_ ''-._ redis_1 | ( ' , .-` | `, ) Running in stand alone mode redis_1 | |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379 redis_1 | | `-._ `._ / _.-' | PID: 1 redis_1 | `-._ `-._ `-./ _.-' _.-' redis_1 | |`-._`-._ `-.__.-' _.-'_.-'| redis_1 | | `-._`-._ _.-'_.-' | http://redis.io redis_1 | `-._ `-._`-.__.-'_.-' _.-' redis_1 | |`-._`-._ `-.__.-' _.-'_.-'| redis_1 | | `-._`-._ _.-'_.-' | redis_1 | `-._ `-._`-.__.-'_.-' _.-' redis_1 | `-._ `-.__.-' _.-' redis_1 | `-._ _.-' redis_1 | `-.__.-' redis_1 | redis_1 | [1] 09 Nov 10:19:23.279 # Server started, Redis version 2.8.17 redis_1 | [1] 09 Nov 10:19:23.279 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. redis_1 | [1] 09 Nov 10:19:23.279 * The server is now ready to accept connections on port 6379 app_1 | Server running at http://0.0.0.0:1337
By default, Fig will start the containers in the foreground, blocking the terminal. To start in the background, add ‘-d’ to the command line.
Docker & Vagrant
As of version 1.6, Vagrant directly supports Docker and Docker containers through a new provider. Additionally, on platforms where Docker is not natively supported (Mac, Windows), Vagrant automatically starts a Linux-based (VirtualBox) vm which can then run a Docker container.
Let’s create a Vagrantfile for our app. We can re-use the existing Dockerfile, and let Vagrant use that:
Vagrant.configure("2") do |config| config.vm.provider "docker" do |d| d.build_dir = "." d.ports = ["1337:1337"] end end
Save the file, and run ‘vagrant up':
==> default: Deleting the container... ==> default: Removing built image... ==> default: Building the container from a Dockerfile... default: Image: d2717466cf97 ==> default: Fixed port collision for 22 => 2222. Now on port 2200. ==> default: Creating the container... default: Name: docker_blog_default_1415529103 default: Image: d2717466cf97 default: Volume: /home/user/dev/docker_blog:/vagrant default: Port: 1337:1337 default: default: Container created: 539ba7edfee50fa0 ==> default: Starting container... ==> default: Provisioners will not be run since container doesn't support SSH.
And once again, our app is available on port 1337, proxied to the container that Vagrant built.
Conclusion
If you’ve made it this far, you now know how to start a Docker container that runs your app, and link that container to other dependencies. Ofcourse this is but a little intro into the exciting world of Docker, which is rapidly changing and growing, and has much more to offer!
The post Docker & PHP: beyond virtual machines appeared first on Touchdown Consulting Services.
Zend Certified Engineer!
Finally got around to doing the ZCE (http://www.zend.com/services/certification/) exam yesterday, and I passed! Click the logo below to see my certification details.
The post Zend Certified Engineer! appeared first on Touchdown Consulting Services.
Deploying PHP applications with Phing (PHPNW11)
Below are the slides of the Phing talk I gave during the PHPNW11 conference. Deploying PHP applications with Phing View more presentations from Michiel Rook
The post Deploying PHP applications with Phing (PHPNW11) appeared first on Touchdown Consulting Services.
Building and deploying PHP applications with Phing (PHPUK2012)
Below are the slides of my Phing talk I gave at the PHP UK Conference 2012. Also, some of the examples discussed during the talk can be found here. Building and deploying PHP applications with Phing View more presentations from Michiel Rook
The post Building and deploying PHP applications with Phing (PHPUK2012) appeared first on Touchdown Consulting Services.
Phing development update
Starting today, I’ll post regular updates on the development of Phing (which is, as you might know, a PHP build tool based on / inspired by Apache Ant). For the past three years I’ve been the lead developer on this open source project, and thus responsible for deciding the direction of the project, integrating the […]
The post Phing development update appeared first on Touchdown Consulting Services.
The road to continuous deployment: a case study
Below are the slides of my talk “The road to continuous deployment: a case study”, as presented at the Dutch PHP Conference in June 2016. I’m planning to dedicate a series of blogposts to this particular topic later this year. It’s a situation many of us are familiar with: a large legacy application, limited or […]
The post The road to continuous deployment: a case study appeared first on Touchdown Consulting Services.
CD: Continuous Delivery or Continuous Deployment?
I’m writing this post to contribute my take on the acronym CD, and the distinction (or perhaps, confusion) between the phrases Continuous Delivery and Continuous Deployment. These phrases are used interchangeably, and sometimes incorrectly, in various books, blogs and talks. And while these software engineering approaches (or disciplines) share a lot, there is one key difference. […]
The post CD: Continuous Delivery or Continuous Deployment? appeared first on Touchdown Consulting Services.