DevSteve

Promoting and Documenting an Open Source Project

2008-09-20T16:16:00.000-07:00

As any of my readers will know, I was part of a team to build the DOMi project, which is an open source PHP object designed for XSL driven websites. The project has had less activity than we would like, and as it is the first project like this for any of us, we are at a loss on how to promote this project.

The three of us genuinely feel that this paradigm is the way of future web development, and all of us use it exclusively in our projects. So there's no lack of love or passion or dedication for this project, we just don't know how to get others to give it a shot and see how well it works.

I recently did some documentation for the project. I hope that helps move it along. We should've done this from the start. It's hard to just give out an object like this and assume everyone can figure out how to use it, especially since some of the internals are doing really odd things. It has some good Javadoc style markup in it, though, which should've helped.

Documentation is a bore, though. A necessary evil that must be endured to get a project on it's feet. I'll be thrilled once this is out there and being used for the next generation of web development. HTML echo needs to be phased out, just like table based designs are being phased out. There is no two ways about it - XSL driven systems are superior to HTML echo driven systems.

Why are RSS feeds so dirty?

2008-09-20T10:14:00.000-07:00

At work, we run a web framework that incorporates some RSS feeds for automatic content generation on one of the modules and a weather display that can be enabled on the main skin. My boss (who wrote the original version over a year ago) and myself (I took the project over about 4 months ago) tried in vain to get a fully W3C compliant XSL / MVC driven system. We ended up giving up due to not being able to get the XSL to generate valid code. I recently looked into it again, now that I understand the XSLTProcessor a lot better than either of usdid a year ago. I managed to correct the output settings to get all of our code valid, but now I am struggling against the horrors of RSS feeds.

Why do so many feeds use noncompliant code?

I am now working on an extension to DOMi that is used for RSS, with the most central tenet of it's design being cleaned up code. My short list of tasks is to get all attributes placed in double quotes, lowercase all attribute and tag names, add required attributes to certain nodes, and convert deprecated tags to more modern ones.

It'll be regex heavy, but I'm quite handy with regex (rightfully owning the xkcd regex shirt), but it shouldn't be too difficult.

Once it's done, it'll go onto sourceforge with the rest of the DOMi stuff.

I cannot wait for Cuil to crash and burn

2008-08-18T11:53:00.000-07:00

In the past ten years, Google has risen from a small startup into one of the most powerful companies in existence. Google knows you better than you know yourself. Google would make a better president than Obama, McCain, Paul or Barr. Google is the new overlord, and I'm happy with that.

So what happens when some minor players who popped up halfway through Google's rise branch off and create a search engine, with the mistaken belief that the dot com bubble never burst?

Cuil is what happens.

Cuil is when a few ex-Google employees start a new company, constantly preach themselves as ex-Google employees, sucker a few investors to drop $33m, and then mismanage themselves into a search engine that has all the usefulness and relevance of Infoseek circa 2000.

Cuil seems to be stuck in the past. The site layout is atrocious, the relevancy is terrible, it has no extra search features (images, maps, news, etc.), and it blows money like it's 1999. Cuil pays for lavish extravagance for all employees, including weekly barbecues, catered lunch daily, gym memberships, on-site doctor, and fridges packed with food and snacks. In principle, this is fantastic, in practice, this is not a good method to run a business, especially when the product is as poor as Cuil.

Cuil is run by thieves. Not in the traditional sense, but in the sense that they were not responsible for Google's success, but they happily took money for proclaiming themselves as ex-Google employees building a Google killer. Cuil is built by a group who backstabbed Google, and the only good thing is that the product is so miserably bad, I know their treachery won't affect Google.

PHP and XSL Part 3

2008-08-14T16:29:00.000-07:00

This is the third entry in a three part series on the advantages of using XML and XSL to generate a web page. Part one will focus on the downsides of echoing HTML through PHP and how an XML/XSL driven system overcomes these shortcomings. Part two will focus on how PHP uses DOMDocument and the XSLTProcessor to generate XML documents and convert the XML into XHTML with an XSL stylesheet. Part three will introduce the DOMi object, a purpose built class that is designed to simplify and speed up the process of building XSL driven websites.

Part 3 - Introducing DOMi

In Part 1 and Part 2 of this article, I explained why the traditional method of HTML echo is on the brink of obsolescence, and why the DOMDocument object is the next logical step in web development. However, if you have used DOMDocument and XSLTProcessor before, you know one major downfall to using those systems - monotonous, tedious tasks to build even a simple object, followed by a hard to remember series of commands to render the XML through the XSLTProcessor.

In Part 3, I will introduce an open source tool that is my bread and butter for XSL driven websites - the DOMi object. DOMi was created purely for XSL driven websites, and as such houses several very useful tools for speeding up the process and briding the gap between PHP data structures, such as arrays, and an XML tree.

How is DOMi used?

DOMi is used in the same way that DOMDocument is used. An instance is created and manipulated. Infact, DOMi has every single function that DOMDocument has. DOMi contains a DOMDocument, XSLTProcessor and DOMXpath, transparently accessed through DOMi. In essence, the following two lines of code are the same...

$Domi->createElement('element');
$Domi->Dom->createElement('element');

The first line attempts to invoke createElement as though it were a member method of $Domi, whereas the second line accesses the $Domi->Dom member property, which is a DOMDocument, and invokes its createElement method. Domi transparently passed the parameters to the DOMDocument object, and will even return it's results. This means that anything written for DOMDocument can be transferred to DOMi with absolutely no issues.

How does DOMi make it easier to build an XML tree?

PHP wasn't designed to manipulate the DOMDocument. It was designed to manipulate arrays and strings and integers. So when you try to use DOMDocument's createElement, appendChild and setAttribute methods to build an XML tree, you will quickly find the process to be extremely time consuming. Building even a simple tree is a labor of many lines and more time than should be required to simply convert one data structure into another.

Enter DOMi::AttachToXml - a method that receives a large variety of data types and converts them to XML and attaches them to the DOMDocument stored within DOMi. AttachToXml accepts two parameters, a data structure and a prefix. The prefix determines what to name the node that is being built, and the data structure is then built into that node, and the node is attached to the root of the DOMDocument. In addition, if you don't want the node attached at the base, you can pass a third, optional parameter that is a DOMNode within the DOMDocument, and the data will be attached as a child node to that DOMNode.

With this system, you can pass an infinitely large multidimensional array to AttachToXml and have a perfect XML tree, ready for the XSLTProcessor. Also - by naming keys in certain fashion, you can even set up your attributes just as easily. With this system, a proper database abstraction layer and a well set up database, you can convert a store's database into an XML tree ready for display in less than 5 lines of code.

How does DOMi render pages?

When DOMi was built for XSL driven websites, there was an obvious need for rendering support. The DOMi::Render method is designed to display a page in one of three formats - HTML, XML or View. The HTML output is the final rendered output, combining the XML and XSL into a web page. The XML output is used for debugging mode to see what the current DOMDocument XML tree looks like - which is essential when you are designing your XSL stylesheets. The View output is a system to convert from one DOMDocument into another and display it on screen as XML. It is used similar to a database view, in that it allows you to alter the real storage structure, and so long as the view is properly updated, any external calls to the XML will still work. This is a great tool for building an API.

To render a page with DOMi, you must first import a stylesheet through the XSLTProcessor::importStylesheet() method. Due to the transparent nature of DOMi, this method can be invoked as DOMi::importStylesheet(). Just as with a regular XSL driven system, importStylesheet requires to be given a DOMDocument that contains an XSL stylesheet, but DOMi has provided tools for importing those stylesheets easily. DOMi::GenerateXSL() receives an array that contains file locations of XSL stylesheets and returns a DOMDocument of an XSL stylesheet that includes all provided stylesheets. This allows for dynamic inclusion of XSL stylesheets, which is useful for CMS skinning.

The stylesheet that is returned by DOMi::GenerateXSL can then be provided to DOMi::importStylesheet and used for final rendering. The last task is to render the page using DOMi::Render(), which accepts one parameter, which is either DOMi::RENDER_XML, DOMi::RENDER_HTML, or DOMi::RENDER_VIEW, and based on what is sent, the DOMi will render the desired results.

A brief look at DOMi

An example of DOMi code to get a list of employees and render on screen is shown below. This script assumes you already have a stylesheet and a function that returns an array of employee data.

//create the document with the root node as 'root'
$Domi = new DOMi('root');

$Domi->AttachToXml(GetEmployeeList(), 'employee')); //attach the array to the DOMDocument

$Domi->importStylesheet($Domi->GenerateXSL(array('stylesheet.xsl')));

$Domi->Render(DOMi::RENDER_HTML);

Conclusion

HTML echo is an old system that needs to be put to pasture. Its limitations are now too great to consider it a viable system for much longer, and a better system has been established. Why continue writing sites in a system that fails in so many regards? Upgrade to an XSL based system as soon as possible. After you get used to it, you'll begin viewing HTML echo in the same light you now view table based designs (I hope, if not, I highly recommend looking into CSS based designs). XSL has a bit of a hard barrier to break. Initially, it seems backwards and worse than HTML echo, but as you get more skilled with it, you begin to realize it's strength, and once you are as skilled with XSL as you are with HTML echo, it becomes painfully clear that HTML echo is a dying system, or at the very least, will be a dying system in the near future.

Sourceforge.net - DOMi

PHP and XSL Part 2

2008-08-13T08:34:00.000-07:00

This is the second entry in a three part series on the advantages of using XML and XSL to generate a web page. Part one will focus on the downsides of echoing HTML through PHP and how an XML/XSL driven system overcomes these shortcomings. Part two will focus on how PHP uses DOMDocument and the XSLTProcessor to generate XML documents and convert the XML into XHTML with an XSL stylesheet. Part three will introduce the DOMi object, a purpose built class that is designed to simplify and speed up the process of building XSL driven websites.

Part 2 - Encouraging DOMDocument

In part one of this series, I covered several shortcomings of the traditional system of using the PHP echo function to display page contents to the user. I highlighted three main areas:

Code cleanliness - issues with inconsistent whitespacing, faulty syntax highlighting, and the need to rectify two different code styles or risk losing proper indentation.
Data organization - PHP has no clean methods for easily navigating complex data structures, and thus issues arise when trying to display the contents of complex data structures
Non headless system - by echoing the display throughout the execution of the script, it is difficult to fully separate the data from the display, which reduces flexibility of design.

All three of these issues are solved in one fell swoop by switching over to an XSL driven system, and this entry in this series is going to explain how to use DOMDocument and the XSLTProcessor to do this.

What is DOMDocument?

DOMDocument is an object built into PHP that is used to manipulate a document that is built under the Document Object Model. The Document Object Model is a syntax and markup language used primarly for web based applications. Anyone familiar with HTML or XML is already familiar with the structure of DOM. Anyone not familiar with the structure of DOM should go read up on it before proceeding, as I will assume you are already familiar with DOM.

For someone with knowledge of DOM, using DOMDocument will prove to be easy. Just as a DOM structure contains a document, nodes, attributes and text, so does a DOMDocument contain a DOMDocument, DOMNode, DOMAttr and DOMText. The following code snippet is a very basic DOMDocument being built with a single root node that contains 3 child nodes.

<?php

$Dom = new DOMDocument('1.0', 'UTF-8');

$Root = $Dom->createElement('root');
$Dom->appendChild($Root);

$Root->appendChild($Dom->createElement('child', 'first'));
$Root->appendChild($Dom->createElement('child', 'second'));
$Root->appendChild($Dom->createElement('child', 'third'));

?>

This will produce the following output...

<?xml version="1.0" encoding="UTF-8" ?>
<root>
<child>first</child>
<child>second</child>
<child>third</child>
</root>

As you can see, DOMDocument usage is not very difficult at all. DOMDocument::createElement is used to create a DOMElement, and DOMDocument, DOMElement and DOMNode support the member method appendChild, which attaches a provided DOMElement to its new parent. However, this series of articles isn't meant to teach how to use DOMDocument, so I'll leave that up to you to learn more than what I've explained here.

What is the XSLTProcessor?

The XSLTProcessor is an object built into PHP that uses XSL, Xpath and XSLT to convert an XSL stylesheet and an XML tree into XHTML. The object itself is very simple, as it just combines the XSL stylesheet and the XML tree. The more complex, and more powerful, part of the equation lies in the XSL stylesheet itself.

If we were to add the following lines to the previous sample, we would load an XSL stylesheet and output the results of the XSLTProcessor's conversion.

<?php

$Xsl = new XSLTProcessor();

$Stylesheet = new DOMDocument();
$Stylesheet->load('stylesheet.xsl');

$Xsl->importStylesheet($Stylesheet);

echo $Xsl->transformToXml($Dom);

?>

As you can see, the XSLTProcessor needs to be given a DOMDocument stylesheet, and the transformToXml method receives the xml tree as a DOMDocument and returns an XHTML output. For basic work, it really is that simple. The complex part is the new part - the XSL stylesheet.

What is an XSL stylesheet?

An XSL stylesheet is an XML document that is formatted with special xsl nodes. These nodes, such as value-of, template, call-template, for-each, variable, param, etc., are used to dictate commands to an XSLTProcessor.

Here is an example XSL stylesheet that uses the previously created DOM tree to display a list of the child nodes.

<?xml version="1.0" encoding="utf-8" ?>
<xsl:stylesheet xmlns:xsl="">
<xsl:template match="/">
<ul>
<xsl:for-each select="root/child">
<li>
<xsl:value-of select=".">
</li>
</xsl:for-each>
</ul>
</xsl:template>
</xsl:stylesheet>

If we break down the document and analyze each node, we can easily see what it is doing.

xsl:stylesheet is the root node to encapsulate the entire stylesheet.

xsl:template is comparable to a PHP function. it is a discreet code snippet that can be executed individually. This template is given the match attribute of "/". Templates can either be given a name attribute or a match attribute. Naming a template allows it to be called at will, and matching a template allows it to be called when the XML tree contains the specified node. In this case, the specified node is "/", which means any root level node. In other words, this template will be called in any situation, and is essentially our starting point for the display.

Next, we put up a simple HTML ul node.

Within the ul node, we use xsl:for-each and a provided xpath to set up a for-each loop. The xpath that was provided is "root/child". Without going outside the scope of this article and explaining xpath, I'll just say that selects all of the child nodes that we created in our earlier document. An xpath is nothing more than a ruleset that is used to match one or more nodes. These rules can be customized in incredible ways to identify a nodelist, and then the contents of the for-each node will reflect each item in the nodelist. This overcomes PHP's foreach shortcoming with only scanning one level of one array at a time, and breaks us from the need to contain derivative or redundant data.

The final xsl node listed here is the xsl:value-of node, which is used similar to PHP's echo. It will take the specified value, which in this case is simply ".", meaning the current node's value, and display it on screen. In this case, it will created an li node and put the value of the node within that li node.

In the end, the XML tree and the XSL stylesheet will combine in the XSLTProcessor to create the following HTML

<ul>
<li>first</li>
<li>second</li>
<li>third</li>
</ul>

The formatting of XSL is identical to HTML (as both use the DOM structure), and thus blends in to create attractive, clean code. Since XSL stylesheets are so commonly used with HTML, almost any IDE that supports XSL will blend the syntax highlighting to keep it nicely viewable. Thus, the first problem with HTML echo is eliminated.

Due to the flexibility of Xpath, complex data structures are navigated with ease, allowing a single xsl:for-each node to loop across data anywhere in the structure, regardless of nesting or location. When you can easily scan and interact with data anywhere in the structure, during the display phase, you no longer have any need to store redundant data in the XML. If you want to derive data from within the tree, you can do it immediately with a single call. Thus, the second problem with HTML echo is eliminated.

Since the display is dictated by an XSL stylesheet that can be swapped out on the fly, your data and display are now firmly separated. The data tree is pure, unencumbered by redundant data, and not forced into any particular look or order. You now have a headless system, and the third and final problem with HTML echo is eliminated.

What I have demonstrated here is just a taste of the power of an XSL driven system. Add in more templates, XSLT functions and a little creativity, and you can turn an XML data tree into any kind of output that you want. In addition to being headless, the data tree is so easily navigated through Xpath, you never, ever need to put in derivative data. If any piece of data can be obtained by looking at the rest of the data, then it is not needed for an XSL driven system.

The final part of this three piece set will focus on the DOMi object. This open source tool is built specifically for XSL driven websites, and contains tools to rapidly transform PHP data types, such as arrays, into XML data trees, while keeping the structure perfectly intact. In addition, it blends in XSLTProcessor to allow quick rendering without the need for intricate knowledge of XSLTProcessor's commands.

PHP and XSL Part 1

2008-08-12T14:37:00.001-07:00

This is the first entry in a three part series on the advantages of using XML and XSL to generate a web page. Part one will focus on the downsides of echoing HTML through PHP and how an XML/XSL driven system overcomes these shortcomings. Part two will focus on how PHP uses DOMDocument and the XSLTProcessor to generate XML documents and convert the XML into XHTML with an XSL stylesheet. Part three will introduce the DOMi object, a purpose built class that is designed to simplify and speed up the process of building XSL driven websites.

Part 1 - Discouraging Echo

#1 - HTML echo loses points for unwieldy whitespacing, unreliable highlighting, and bouncing back and forth between C style code and DOM style code.

The modern convention for PHP based websites is to use the echo function to send output to the display. This is problematic for many reasons, yet only a handful of sites use a system other than the tried and true echo system. One of the main drawbacks to using echo is the need to jump back and forth between two vastly different codebases within the same document.

In a single document, we often go into and out of PHP and HTML frequently. Occasionally, these calls are nested within one another (for instance - in an HTML list created by a PHP foreach). We can echo out the HTML as well, but that causes some whitespacing issues with awkwardly formed strings being passed to echo.

In most IDEs, we also lose our syntax highlighting on the HTML inside the echo. In addition to the whitespacing, HTML uses DOM style formatting (nested nodes, attributes, etc.), while PHP uses C style formatting (curly braces indicate scope changes, etc.), and it is occasionally hard to rectify the differences while keeping indentation intact.

#2 - HTML echo loses points for requiring derivative data and clumsiness in scanning a large, deeply nested data tree.

The next drawback of HTML echo lies in derivative data. If you had a multidimensional array, where each element contained an employee task, and each task was given a category, you would have a difficult time getting a count of tasks by category. You would need to run a foreach loop across the array and keep a count of how many tasks fell into each category.

The main problem with this foreach loop is that a count of tasks by category is derivative data from a list of all tasks. Rather than each piece of data being unique and necessary, you now have muddied up the data with something that can be figured out by looking at the rest of the data. If we had a system where extracting deeply nested data were incredibly easy, then we could have PHP only pull unique, essential data and allow the display tools to derive anything additional that is needed.

A related issue is PHP's difficulty in extracting information from a variety of places within an array in a single call. One project I had some time ago involved a multidimensional array containing employee groups. Each group had either subgroups or an employee list, and the depth of an employee was variable, depending on how many subgroups an employee group had. Some employees were fairly shallow, for instance /sales/Ben, whereas some were fairly deep, such as /operations/it/seo/blue/Logan. Because of this, PHP couldn't get me a list of all employees very easily, as a foreach loop only scans one level at a time.

#3 - HTML echo loses points for forcing data and display to be bound together.

The final point I would like to make in this first entry is on how difficult it can be to alter the designs in a page that is using an HTML echo system. When using HTML echo, you are often linking the data and the display in hard to separate ways. While there are some systems that do a fairly good job of keeping data processing away from the display, most of them are not complete and will still leave traces of data touching the display.

When the data and display are intertwined, it becomes difficult to create a truly headless system - one where the display can be changed at will without ever touching the data. A headless system has many advantages, such as ease of converting to an RSS feed, an API or vast site redesigns. When you have to alter the data, even just a bit, to alter the display, you lose mobility and flexibility.

In brief, HTML echo systems were good for awhile, but we have moved beyond that. Just as CSS overtook tables when designing a site, the time is right for XSL to overtake HTML echo. HTML echo is ugly to code, requires repetitive data, is difficult to extract data, and is difficult to change the design.

Part 2 will focus on using DOMDocument and the XSLTProcessor to alleviate these problems. I will show you how you can have PHP build a pure data tree and use XSL to convert that data tree in to a display.

Part 3 will focus on the DOMi object, which is available at sourceforge.net, and how that smooths out the kinks in DOMDocument and integrates XSLTProcessor and DOMXpath into a single, powerful object.

The launch of DOMi

2008-08-11T15:09:00.000-07:00

Earlier today, the first release of the PHP DOMi object went up on sourceforge. This object was one created by a group of people of which I am a member. This is my first foray into contributing to the open source community I so vocally adore. This object was built with XSL driven websites in mind. It is a tool to ease the transition between PHP data structures and XML data trees, and allow for quick rendering of the XML tree through the XSL stylesheet.

I'm quite excited about all of this. I love XSL driven websites and would like to see this technology overtake traditional HTML echo websites as soon as possible. It is far neater, far cleaner, far more powerful. On top of that, I really want to give back to the open source community and have long been a proponent of knowledge and information being free for all, and now that I have a portable, stable object that I feel can help many people, the first thing I wanted to do was make this available to anyone and everyone.

If you are a PHP developer, I ask that you check out DOMi. The documentation is a bit sparse right now, but it's not hard, and I'm always available to answer any questions. The object is fairly simple, though, so it shouldn't be hard to figure out how to use it.

Infinite Goods Economies and Developers

2008-08-07T15:05:00.000-07:00

Techdirt has done a wonderful job of shaping my thoughts on infinite goods versus scarce goods. The brief on this concept is simple - in the digital age, when information can be infinitely duplicated and distributed at virtually no cost, the rules of economics must change. All goods now break down into two categories, infinite goods and scarce goods. In the music industry, the infinite good is the music, the scarce good is the live performance. In the movie industry, the infinite good is the movie, the scarce good is the theater experience.

In my discussions on this economic concept, one thing that I have been asked repeatedly is on the subject of how a developer fits into this economy. My trade is in code, which is an infinite good, so how do I promote the scarce good by providing the infinite good, and what exactly is the scarce good for a developer?

The answer is simple - I am the scarce good. My code is infinite. A PHP class that I created for advanced DOM manipulation is an infinite good that can be copied and reused and altered infinitely at no cost. The skill that created that class cannot be copied. A PHP class that I created to parse an XML document into a series of SQL calls for installing or updating a CMS is readily available to all who know where to look, but I am only one person with a limited amount of time.

I am an extreme proponent of open source technologies, and I have been asked how I can benefit from giving my code to anyone who wants it. While it is true that I don't earn money off of these objects, I earn money by increasing my value as a developer and thus increasing the pay range I can reasonably expect for a job. Each time I contribute a good piece of code to the open source community, I not only help the community that has given so much to me (I'm currently using Firefox, chatting on Pidgin, coding in Quanta and using Linux Ubuntu 8.04, all open source), but I help myself tremendously by bolstering my reputation as a developer.

So the infinite goods economy doesn't apply only when the scarce good is a tangible object, as I have heard before. A scarce good is simply anything that cannot be duplicated easily, whether that is a theater, a musician performing live or the skill of an individual developer. Next time you run into a situation in your own industry where you see a good that can be infinitely duplicated, just think of how you can use that to promote the things that can't be infinitely duplicated. Once you have that figured out, you have taken the first step towards adapting to the economy of the future.

Can Google's Knol Dethrone Wikipedia?

2008-07-23T18:23:00.001-07:00

Google launched knol today, a Wikipedia like site that allows users to submit articles on almost any subject they choose. This is a clear shot at cutting into Wikipedia's domination of the Web 2.0 encyclopedia market, but how well can it really work?

Knol offers a few things that Wikipedia doesn't, with one of the most major differences being that an author can earn ad revenue from the article he or she has written. Some parts of this are good, some are bad. It lets the authors earn money, but also has authors vying for their chunk of the pie.

While it is good to allow an author to earn money for his work, it also sets up a situation with multiple articles on the same subject, which dillutes any article from ever being truly comprehensive. Wikipedia is highly beloved by many precisely because of the fact that it aims to have only a single article per subject. With Wikipedia, you won't find half the story on one page and half the story on another page too often, and in those situations, you'll almost always find links between the pages. With Knol, authors won't be encouraged to link to other authors, because that is akin to sending customers to your competitor.

When you have authors fighting against one another, rather than collaborating, you won't get the level of detail and thoroughness for which Wikipedia has become known. Instead, you'll have 15 articles on Linux, each one both trying to add something new and copy what information the others have but they don't.

Knol aims to pay each author for the ad revenue generated, but for some, this may be short term profit over long term losses. If I spent time trying to get the best article on a tech subject in there, I'm losing time where I could be absorbing the single Wikipedia article on this subject and putting this new found knowledge to use. This new found knowledge can advance my skills as a developer, and what I learn can be transferred back into the community. Along with this knowledge returning to the community, my increased knowledge increases my value as a developer and allows me to earn more money in the long run. I am fighting where I could be working together with others to bring us all up together.

Another thing that Knol lacks over Wikipedia is an established article base. Wikipedia has exponential growth because it is a Web 2.0 site that employs the most fundamental of Web 2.0 methodologies - the community as the author. The larger Wikipedia becomes, the more audience it will have, and that increased audience will make Wikipedia even larger. Knol comes into the game with no existing knowledge base and no reason to go there other than to post articles.

While I love almost everything Google, I think they took the wrong plan here. Rather than trying to fight Wikipedia, they should've tried to partner with Wikipedia. Use Google's skills and Wikipedia's skills to mutually enhance their products. I think both would've been better served to set in place a semantic search feature where Google can parse Wikipedia's pages to find an answer to a question, similar to TrueKnowledge.

Stop protecting the children!!

2008-07-23T14:40:00.000-07:00

It seems that almost every week a new story comes out about a company trying to 'protect the children', and in nearly ever instance, their protection is misguided, self-serving, redundant, useless or all of the above. In the past few weeks we have seen quite a few, such as cable companies blocking child porn and google maps is used by child predators. Of course each time a technopanic arises, someone is there to 'protect the children' by enacting measures against it, and each time the tech community is up in arms about how pointless and counterproductive these measures really are.

Sure, it's nothing new to use fear tactics and create solutions to fake problems to bolster your reputation, but if these measures are allowed to take hold in greater numbers, we risk losing all of what the internet means. No sane person in the tech community advocates keeping child porn online, and anyone out there to 'protect the children' who thinks we do is quite mistaken. All we are suggesting is that these measures focus on eliminating the problem, rather than a symptom, or worse yet, making those problems harder to find.

Not too long ago, New York considered making it illegal to incriminate yourself online. Such a law is not only pointless, but actually makes law enforcement more difficult. As Techdirt noted in their commentary on this incident, by making it illegal to incriminate yourself, the NYPD is making it harder to prosecute crimes. Blocking child porn is the same concept. If you make it more difficult for everyone to access it, you only make it more difficult to track down and eliminate.

All of these illegal activities will still be happening, whether they are blocked or not, but by blocking access, you leave the only people likely to find it as those actively seeking it. If someone who wasn't not seeking child porn came across some, they would be more likely than a non law enforcement agent who was seeking it out to contact law enforcement.

The whole 'protect the children' call to arms is merely a self serving tool of organizations and individuals who want to advance their career through short sighted policies that serve no true purpose. If they want to protect the children, they would be concerned with finding those who create child porn and bringing them to justice, rather than simply making it harder to find the child porn.

The Forgotten Side of Consistency

2008-07-18T11:56:00.000-07:00

Consistency is a subject in which I often find myself. Consistent coding styles, development standards, and common libraries and frameworks can help a team of developers work as a single entity. If all functions are indented the same and all variables are given a similar capitalization convention, it becomes easier to rapidly scan through code. When you aren't wondering what scope you are in, you can focus on figuring out what is happening.

However, there is one piece of consistency that far too often gets overlooked - function and variables should be named in accordance with what they do, and all functions that are similar in nature should be named in a consistent pattern.

For instance, I recently worked on an install script whereby a PHP script processes and XML document and modifies a MySQL database to match what is shown in the XML document. In this script, I had no less than six functions that converted an XML node into a PHP array for later processing.

These functions had names such as "ProcessTables", "ConvertFields", "ConfigureKeys", "GetRecordsConfig". These names are all quite different, yet they apply to functions that are all the same. This is bad practice for one simple reason - when someone who is unfamiliar with your code looks at it, you have hindered their ability to quickly figure out what is happening. With that in mind, don't forget that if you haven't worked on a script for a few months, it might as well have been written by someone else.

Here are a handful of tips to help you with consistent function naming:

Always do pseudocode prior to writing any actual code. A hasty developer is a bad developer. The sooner you start writing, the longer it will take. There are a million reasons why you should pre-plan your code, but to add onto that lengthy list, it makes it a lot easier to set up function naming conventions.
Write an incredibly generic description of what each function does. All of those functions I listed above do the same thing - convert an xml node to an array. The node they work on and how they do their conversion may change, but at their core, they just convert an xml node to an array. When I have a list of my functions, I can go back and give each one a generic description and see which functions essentially do the same thing.
Set up a naming convention based on it's generic description. Once you have broken all of the functions down into categories based on description, you can set up a naming convention that will be used for each one. For instance, for all of my functions that convert an xml node to an array, I set up the convention of 'Convert*******Xml' and simply slipped in the description of the node. 'ProcessTables' became 'ConvertTablesXml' and 'ConfigureKeys' became 'ConvertKeysXml'.
Create a legend in the comments. Once you have set up a naming convention, it is a good idea to mark off a section in the comments that lists the convention and where each one applies. This will help you later on if you need to add more functions and need a reminder of what the convention is.
Stick to the convention. The work doesn't stop once the initial development is completed. If you go back to the script and have to make additions, be sure you continue to follow this convention. By using a convention on only half the code, you are only doing half of what you should be doing.

Framework City - the most dangerous place on Earth

2008-07-17T17:20:00.001-07:00

A well written framework is a ridiculously powerful tool for simplifying development. It allows a novice programmer to make some nice scripts and a veteran program to hammer out fantastic scripts in short order. A powerful framework practically writes the software itself.

A good PHP framework provides easy to use wrappers for database connectivity, accepting input, rendering output, and authenticating users. A good module class can make the most complex function calls so easy you may forget how complex it really is. There is nothing like developing a fantastic framework to make all of your projects easier to handle.

Except, perhaps...finishing a project.

Welcome to Framework City, the developer hellhole. Here in Framework City, you will be stuffed to the gills with fantastic ideas on upgrading your framework to make it even easier. You'll be so contented with building the ultimate framework, you won't even notice the increasingly annoyed client who wants results, not promises that once you're done with the framework, the site will build itself.

Writing a framework is a never ending task. There is always a better tool to implement, always a new library to implement, always a new way to make your life easier. Developers write frameworks for themselves. Clients paid money for a site for the client. Getting stuck in Framework City is a blissful paradise, but only for the developer who keeps making life easier on himself.

Next time you find yourself wanting to upgrade your framework, keep Framework City in mind. Avoid the pitfalls of getting mired in making better tools, and put the tools you already have to work. You'll find your client is more pleased with the increased results, and you'll be pleased with a less angry client. The framework can always be upgraded between projects. As you do a project, make a list of annoyances with the framework and handle all of them at the end. This also prevents you from doing changes on a whim and later realizing that was a bad idea.

So while frameworks are truly a gift from the gods, they are also a very tempting quagmire where you'll kill your career in perfect bliss.

Absolute Paths and Relative Paths

2008-05-22T17:23:00.000-07:00

Lately, I've had to grand experience of fighting my own terrible code. I've been doing PHP work for about 18 months now, and my early work was pretty bad. In the past year, I've made great strides in improving my code, but I still have a lot of old work that is still in place. This week, one of my main projects is setting up a local development environment so I can freely work on sites without fear of disabling an active website.

One problem I keep encountering was my old tendency to declare absolute paths. I would define all includes and links by giving the exact file location, rewritten each time. Nowadays, I would never be caught doing something so foolish as rewriting the same file path over and over. Now, I would define a constant at the very beginning of an MVC website, and for the rest of the time the script is executing, in pulls files in relation to that constant.

By setting things up in this manner, I'm using the best of both worlds. I have the security of using an absolute path that can never be confused on a file's location, and the flexibility of a relative path that can be relocated and still functional.

The best way that I've yet found is through a series of constant definitions at the start of every site:

define('ROOT_DIR', dirname($_SERVER['SCRIPT_FILENAME']) . "/");

define('HTML_ROOT', "http://" . $_SERVER['HTTP_HOST'] . "/" . str_replace($_SERVER['DOCUMENT_ROOT'], '', ROOT_DIR));

The first definition defines the root directory that the site will be executed from. By placing the index.php file in the main directory and then using the dirname() function, you can locate the full path for the main directory. Unlike using the $_SERVER['DOCUMENT_ROOT'] variable, this can account for a site that isn't at the root level of a site.

The second definition defines what should precede all URLs listed within the site. By putting that constant infront of all CSS declarations, Javascript declarations, images, and links, you can be assured that the file will always be properly found.

For instance, if your main domain is http://my.domain.com/user/directory/site/, the CSS link of /styles.css will go outside the site folder and go back to the root folder. However, adding this constant before the call will change it to http://my.domain.com/user/directory/site/styles.css, without you worrying about it.

As you can see (and see even better if you try it), by declaring the absolute path constants at the start of the site, and basing those declarations on a series of function calls that analyze the current page, you can make use of both absolute and relative paths. The constants are defined on a fluid, yet reliable, method, and those constants are absolute and safe. A site built with this in mind can be moved to any domain at any time and every line of code will still work flawlessly.

The joys of setting up a development environment on Linux

2008-05-21T16:51:00.000-07:00

I spent a large portion of the day today setting up a proper development environment. My job has needed one for some time, but us developers are notoriously overworked. Luckily, my current project (a new module added into the client section) gave me a perfect excuse to build up a development environment. I have had plans for some time to set up a proper CVS repository, local Apache servers, and make good use of our development server.

In the 14 months that I have spent at my current job, one thing that I've learned is that attempting to make changes to pages in a live environment is a pain in the ass. I have about a two minute window where I can have glitches and errors streaming on the page before I get a phone call or special visitor complaining about a downed site. Even from day one, I knew this was a bad idea, but only recently have I really been able to do anything about it, and here's what I did:

First off, a local apache build is essential for a PHP developer. I run Kubuntu 8.04 and through a single command, I got my entire server set up and ready to go. Open up a terminal and type the following to set up Apache2, PHP5, MySQL 5 and PHPMyAdmin:

sudo apt-get install php5 php5-mysql phpmyadmin apache2 mysql-server-5.0

If you want some additional PHP modules, you can throw in a few more for good measure. I added php5-xsl, php5-gd and php5-curl. Once this processes, typing 'localhost' into the browser URL bar will get you to view your /var/www directory as a website. This is the first critical step to a proper development environment.

Next up, to get phpmyadmin properly viewable, edit the /etc/apache2/apache2.conf file and add a single line at the end:

Include /etc/phpmyadmin/apache.conf

This will allow phpmyadmin to become accessible at http://localhost/phpmyadmin/.

So now that you have a local apache server, you can begin making sites in a development environment. However, most sites require database information to operate, so you have to sync up your live mysql server with your local mysql server. This can be done manually through export/import of sql files, or you can set up an automated synch tool. Personally, I set up a crontab and a series of SOAP calls that authenticates itself to my remote server, gets the database's table list, and then goes through each table, gets the structure, deletes the local table, rebuilds the local table, gets the table data, and populates the table. This operates every night so I don't even have to worry about my data being matched up.

The next step, one that is not critical, but I find to be useful, is to set up your hosts file to access the site in an easy fashion. This is especially useful if your site uses a lot of absolute paths for it's links and relocating from "http://www.domain.com" to "http://localhost/domain/" would cause linking issues.

If you want to set it up so that typing in "http://localdomain/" redirects to /var/www/domain, follow these steps:

Edit the /etc/hosts file and add a new entry. You should assign a new IP address for each entry, and the text field will be what you want to type in. For this example, add in "localdomain" to the entries listed at the top.

Next, edit /etc/apache2/sites-available/default to add in the new virtualhost. At the end of the page, add the following lines:

<VirtualHost *>
ServerName localdomain
DocumentRoot /var/www/domain
</VirtualHost>

The first step, editing the hosts file, adds an entry on what the local machine may be referred to. Adding in 'localdomain' tells the local hosts file that when that site is attempting to be accessed, redirect to the local server.

The second step, editing the site file, tells Apache what to do if it is asked to go to localdomain.

The final step is to restart the apache server and then test the site.

Now that you've done all of this, you can safely build and tinker with a development version of a site, rather than try to toy with the live version that people are using. Combine this with a proper CVS setup, and you can have a team of developers working in a coordinated, organized environment with well managed projects.