In an attempt to move my projects towards this model, I ended up running into several problems.

You assign “users” to “groups” by doing the following:

GRANT role_group TO role_user;

Where role_group is the name of the group role and role_user is the name of the user role.

The assignment of “users” to roles plays a particular importance in permissions assignment. Typically, the webapp model has been such that it connects as one user to the database to perform SELECT, INSERT, UPDATE, and DELETE queries. Thus, it was easy enough to simply:

GRANT SELECT, INSERT, UPDATE, DELETE ON TABLE TO user;

With PostgreSQL, you have to execute the above command for each table, sequence, and other object type in order to grant access to your tables. If you have 50 tables, that’s not a tedious task, but it’s certainly annoying. Now envision you need to grant access to your developers and/or analysts to be able to perform the same queries. Share the username and password of the webapp? This violates a lot of industry practices and will be very difficult to trace back which user accidentally (or maliciously) mucked with the data in an inappropriate fashion.

So, you’re stuck executing the same 50 queries however many times you need to grant access to the database. Now it’s laborious and you start asking “What’s the better way to do this?”. Roles. If you executed the same 50 queries above on the “group” role, then to grant another user access to those same tables, you execute only the single query to add them to the role and BAM! They have access….or so you thought.

This is ultimately what this post was about. The PostgreSQL GRANT documentation doesn’t include anything indicating that there’s a bit of a caveat with GRANTing access to group roles.

When you implement the solution above, you may find yourself scratching your head when your user, a member of a group that has access to the schema and table you’re trying to query, receives the following error message:

ERROR: permission denied for schema cfegroup

Where cfegroup is the name of the schema; or, you get the following error message:

ERROR: permission denied for relation agency

Where agency is the name of the table, then you have stumbled upon the same problem I had. Eventually, I found the answer still on the PostgreSQL documentation, but not in the GRANT section. Rather, the problem is indeed a permissions problem because your user is not acknowledging the role permissions. According to the CREATE ROLE documentation:

These clauses determine whether a role "inherits" the privileges of roles it is a member of.
A role with the INHERIT attribute can automatically use whatever database privileges have
been granted to all roles it is directly or indirectly a member of. Without INHERIT, membership
in another role only grants the ability to SET ROLE to that other role; the privileges of the
other role are only available after having done so. If not specified, INHERIT is the default.

Thus, you simply need to make sure that your database “user” role has INHERIT on it. Once you have verified this is the case, the user should not have a problem accessing your objects.

In order to undertsand why PHPUnits mocking was not working, and what I had to do to fix it, it’s necessary for me to explain the concept of mocking. Assume that you have 2 classes, a controller and a DAO. Assume the controller talks to the DAO to make queries against the database. When running unit tests against the controller, you don’t want the DAO to actually connect to the database, much less worry about whether the logic in the DAO is valid or not. Mocking will help emulate the DAO, but because it’s a runtime-configurable object, you can tell it what to expect and what to do if the conditions are met. This helps you present assumptions that everything is handled correctly by the DAO, leaving the controller tests to truly test only the controller.

Now, typcially you will not actually mock an object, but rather mock an interface. The purpose behind this is important. When you mock an implemented class, typically the mock will execute the logic already programmed into that class, especially if it’s declared final. When mocking an interface, none of the methods are actually implemented, so it becomes a true mock - everything is 100% runtime-programmable.

Now, when I upgraded the version of Zend Framework I was using, this is the error I started getting:

Fatal error: Call to a member function quoteInto() on a non-object

This was for the following code:

public final function deleteById($id)
    {
        try {
            $where = $this->_dao->getAdapter()->quoteInto('id = ?', $id);
            $this->_dao->delete($where);
        } catch (Zend_Db_Exception $e) {
            throw new Cfegroup_Db_Exception('Unable to delete record from database.', $e->getCode(), $e);
        }
    }

Note that $this->_dao is an instance of a Zend_Db_Table_Abstract object as a StaticDao. Now, when stepping through the code, everything seemed okay. In fact, the Zend_Db_Table_Abstract object held the instance of the Zend_Db_Adapter_Abstract that it was supposed to hold, but the getAdapter() method did not return it.

The following subtle difference appeared between Zend Framework 1.6.1 and Zend Framework 1.8.x:

<     public function getAdapter()
---
>     public final function getAdapter()

The top is the newest version of the file and the bottom is the old. The final declaration had previously prevented the mock from being able to extend it and program it, whereas now the getAdapter() method is free to override and re-implement the method. However, when it’s not progarmmed via runtime code, it fails to do anything - not even revert back to it’s originally intended purpose.

In short, I had to add a method to the StaticDao that declared the getAdapter() method as final. Once that was done, the mocks stopped overriding the method and the unit tests continued working.

I’m running Gentoo Linux AMD64 on a dual-screen monitor utilizing Xinerama. I ran into a problem when trying to start the installer for Zend Studio.

Preparing to install...
Extracting the JRE from the installer archive...
Unpacking the JRE...
Extracting the installation resources from the installer archive...
Configuring the installer for this system's environment...
 
Launching installer...
 
'SWING' UI not supported by VM.  Reverting to AWT.
Invocation of this Java Application has caused an InvocationTargetException. This application will now exit. (LAX)
 
Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: 1
	at sun.awt.X11GraphicsEnvironment.getDefaultScreenDevice(Unknown Source)
	at java.awt.Window.init(Unknown Source)
	at java.awt.Window.<init>(Unknown Source)
	at java.awt.Frame.<init>(Unknown Source)
	at java.awt.Frame.<init>(Unknown Source)
	at com.zerog.ia.installer.LifeCycleManager.g(DashoA8113)
	at com.zerog.ia.installer.LifeCycleManager.h(DashoA8113)
	at com.zerog.ia.installer.LifeCycleManager.a(DashoA8113)
	at com.zerog.ia.installer.Main.main(DashoA8113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.zerog.lax.LAX.launch(DashoA8113)
	at com.zerog.lax.LAX.main(DashoA8113)
This Application has Unexpectedly Quit: Invocation of this Java Application has caused an InvocationTargetException. This application will now exit. (LAX)

Some initial Googling indicated that I needed to install i386 compatible libraries. The odd thing about it was I had done so - probably several months ago in fact. However, I ultimately managed to find the temporary solution on the Sun web site, in that this was a Java bug relating to Xinerama. Basically, try running the installer on screen 0:0 instead of 0:1.

Posting Reveals Loading and Hangs

One of the first problems I encountered when posting the user to the uPay site was that the site would “hang”. In essence, I would get a prompt of Loading, please wait…, and the site would go nowhere. When checking my JavaScript console, I got the following error message:

Error: document.getElementById("indexForm").submit is not a function
Source File: https://secure.touchnet.com:8443/C21490test_upay/web/index.jsp
Line: 39

This one took me a little while to debug, since the exact same JavaScript function that I used on my site to redirect the user to the uPay system basically had the exact same code. The solution came out to be painfully obvious - a form button with the name of submit. Since I had a form element with the same name as the function I was trying to call, JavaScript’s DOM was trying to make a function call out of an element in the form. In short, if you have a button on the page to submit users who have JavaScript disabled, make sure to name it something other than submit.

Access Denied

If you’ve read the TouchNet uPay documentation carefully, you’ll notice that the VALIDATION_KEY format is a base64 encoded md5 hash of the following (without the brackets):

[POSTING_KEY][EXT_TRANS_ID][AMT]

So, PHP developers out there may be thinking their solution is as simple as:

public static function createValidationKey($postingKey, $transactionId, $amount)
{
	$currency = new Zend_Currency('en_US');
	$currency->setFormat(array('display' => Zend_Currency::NO_SYMBOL));
	$tmpString = $postingKey . $transactionId . $currency->toCurrency($amount);
	$hash = md5($tmpString);
	return base64_encode($hash);
}

Sadly, this does not work. When the user is sent to the uPay site, they’re displayed the message You do not have access to this site.. This took a bit of looking into, and ultimately deals with how PHP handles md5 hashing and base64 encoding.

PHP’s md5 hash is hexidecimal and is returned as a string. base64_encode in PHP expects ASCII text to encode. When you pass the md5 hash directly into base64_encode, it doesn’t complain because technically what it’s operating on is nothing but ASCII text. However, you have to pack() the md5 hash prior to encoding in order to have base64_encode to generate the same data expected on the uPay system. The resulting solution is:

public static function createValidationKey($postingKey, $transactionId, $amount)
{
	$currency = new Zend_Currency('en_US');
	$currency->setFormat(array('display' => Zend_Currency::NO_SYMBOL));
	$tmpString = $postingKey . $transactionId . $currency->toCurrency($amount);
	$hash = md5($tmpString);
	// From http://www.bursar.gatech.edu/marketplace/EPay_System_Validation_Key.pdf
	$asciiHash = pack('H*', $hash);
	return base64_encode($asciiHash);
}

My Googling yielded a helpful PDF with just what we were looking for. Thanks also go out to Ben Floyd for working with me on finding this solution.

    public function itemExists($item)
    {
        $className = get_class($item);
        foreach ($this->_items as $lItem) {
            if ($className == get_class($lItem) &&
                $className::compare($item, $lItem) == 0) {
                return true;
            }
        }
        return false;
    }

In a nutshell, being able to call a static function for a class through a variable won’t be around till PHP 5.3.0. This is one of my pet peeves about PHP - not having all the OO features that other real OO languages already have.

Now, onto the second facet I hate - not being able to overload functions:

    /**
     * Performs a comparison between two objects of this type to determine if they're equal or different
     * 
     * Currently, the return value returns -1 if $a->_id < $b->_id, otherwise +1 if $a->_id > $b->_id. 
     * It returns 0 if the ids are equal.
     * 
     * @param   Module $a
     * @param   Module $b
     * @return  int
     */
    public static function compare($a, $b)
    {
    	if ($a->getId() == $b->getId()) return 0;
    	return ($a->getId() > $b->getId()) ? 1 : -1;
    }
 
    /**
     * Performs a comparison between this object and another object to determine if they're equal or equivalent
     * 
     * Refer to the rules of the static <code>compare()</code> function for the results of the comparison. 
     * 
     * @param   Module $object
     * @return  int
     */
    public function compare($object)
    {
    	return self::compare($this, $object);
    }

The result when running unit tests:

Fatal error: Cannot redeclare Group::compare()

*fumes*

The solution more or less was me plugging into the vast functionality that Dojo has in each of its objects and some of its base methods. Because Dojo interacts and operates with the DOM so well, it’s very easy to write JavaScript code that will take what Dojo has already manipulated and continue to manipulate it some more.

I found a really explanatory blog about Dojo DND and this started my basis of trying to implement a solution. However, their blog article didn’t address how to submit all of that data through a form - it was simply a live update and utilization of DND components.

What I ended up doing was separating my form (form created using a dijit.form object) from my DND object blocks. I then had my Dojo buttons that subscribed onClick to a custom JavaScript method which gathered the list of “Subscribed” objects, populated the form with new input objects, and then submitted the form. Let’s look at the DND objects first.

<div class="view" width="100%" style="padding-left: 100px;">
	<div class="catalogContainer">
	    <h2>Available Chapters</h2>
	    <ul dojoType="dojo.dnd.Source" accept="normal,subscribed"
	        id="catalogNode" class="container">
	        <?php foreach ($this->chapters as $chapter): ?>
	           <li class="dojoDndItem normal" dndType="normal" id="<?php echo $chapter->getId(); ?>">
	               <?php echo $this->escape($chapter->getTitle()); ?>
	           </li>
	        <?php endforeach; ?>
	    </ul>
	</div>
 
	<div class="cartContainer">
	    <h2>Chapters Subscribed</h2>
	    <ol dojoType="dojo.dnd.Source" accept="normal,subscribed"
	        id="cartNode" class="container">
	        <?php foreach ($this->module->getChapters() as $chapter): ?>
	           <li class="dojoDndItem subscribed" id="<?php echo $chapter->getId(); ?>" dndType="subscribed">
	               <?php echo $this->escape($chapter->getTitle()); ?>
	           </li>
	        <?php endforeach; ?>
	    </ol>
	</div>
	<div class="clear"></div>
	<ul class="actions">
        <li>
            <button dojoType="dijit.form.Button" type="button" iconClass="dijitEditorIcon dijitEditorIconDelete"
                    onClick="window.location.href = '<?php echo $this->url(array(
                        'moduleId' => $this->module->getId()), 'adminModuleView'); ?>'">Cancel</button>&nbsp;
        </li>
        <li>
            <button dojoType="dijit.form.Button" type="button" iconClass="saveIcon"
                    onClick="submitModuleChaptersAction()">Save</button>
        </li>
    </ul>
</div>

What you see above is the two DND divs (catalogContainer and cartContainer) and an unordered list of buttons (see ul class=”actions”). This is not wrapped in a form, but is almost entirely driven by the Dojo engine. In short, I am able to drag-n-drop objects from either the catalogContainer or cartContainer. I click either the Cancel or the Save button to continue.

The Save button is the key - it’s onClick method calls submitModuleChaptersAction(). Let’s take a look at what that’s doing.

/**
 * Submits the form for updating the list of available chapters
 *
 * This method gathers the list of chapters that are subscribed (in order) and drops them
 * into a Dojo form for submission to the action controller. 
 *
 * @return void
 */
function submitModuleChaptersAction() {
    var chapterSub = dojo.byId('cartNode');
    var chapterSubDndSrc = new dojo.dnd.Container(chapterSub);
    var chapterarray = new Array();
    var idx = 0;
    chapterSubDndSrc.getAllNodes().forEach(function(node) {
        chapterarray[idx++] = node.id;
    });
    for (var i = 0; i < idx; i++) {
        var inp = dojo.create('input', { type: "hidden", name: "chapterId[" + i + "]", value: chapterarray[i] }, 
            dojo.byId('moduleChapterForm'), 'last');
    }
 
    dojo.byId('moduleChapterForm').submit();
}

This method fetches each of the nodes in the cartNode Dojo ID (aka cartContainer) and for each of them creates a new DOM object (a hidden input element). This DOM object is appended last to the form, and then the form is submitted. I recognize that the JavaScript code could be cleaned up, and will likely do that soon.

Oh yea, what form?

<div dojoType="dijit.form.Form" name="moduleChapterForm" jsId="moduleChapterForm" id="moduleChapterForm" 
        method="post" action="<?php echo $this->url(array(), 'moduleChaptersSave'); ?>">
    <input type="hidden" name="moduleId" value="<?php echo $this->module->getId(); ?>" />
</div>

This is the dijit form that I use to add new input elements. Likely I could have used a regular form, but I don’t really have the time to test it.

Hope this helps!

Well, Vimperator recently released an upgrade to 2.0. This upgrade really overhauled the performance and indicates that the developers are serious about making sure this plugin is useful for its audience. It did, however, break compatability with one of my other plugins, Tab Mix Plus.

Tab Mixx Plus is a great tool in that it can help manage your tab bar and do cool things such as make all tabs a fixed width and spill tabs over to other rows (so that if you have 40+ tabs open like me, you can still stay somewhat organized). Additionally, if you open new tabs from a link inside an existing tab, the color of the tabs will change so you can see which are related. However, upon reading about Vimperator 2.0 and Tab Mixx Plus, I found that the Vimperator team doesn’t prefer Tab Mixx Plus because of the way it interfaces with Firefox.

Their alternative is Tab Kit. In addition to being more standards compliant in terms of how it interfaces with the tabs in Firefox, it also has more features than Tab Mixx Plus and also is more efficient. This is another plugin that I’ve found helps improve Firefox performance, since I was hell bent on reducing the amount of memory Firefox consumed on my 4GB system.

Hope this helps.

Flash forward to today and we find that Firefox 3 is a bloated monster, especially on anyone’s computer that has more than 2GB of RAM. Why? Well, everything points to the caching that Firefox does behind the scene’s to make it have accellerated performance. However, after a certain point, this performance drops not just the browser, but the entire system. The caching algorithms by default use a percentage of total memory. Thus, the more memory you have, the greater initial percentage taken up by the caching system. So, on a system with 4GB of RAM (such as mine), I easily see Firefox consuming > 1GB of RAM - even with only 12 or so tabs open and things like Flash blocked.

I believe I have found a Firefox plugin that has managed to bring that memory bloating under control. The plugin is called RAMBack. So far on my Linux system, Firefox has gone from consuming ~25% of memory (~1GB) to just 4-6% (roughly between 175MB and 225MB). I’m going to try this plugin out on different platforms and see how it compares, and I encourage anyone else to do the same if they want to see immediate performance.

For reading e-mails, I use courier-imap and courier-authlib. They connect to a PostgreSQL server to check the table of virtual users to see where it should look for my maildirs.

Now, the problem is it does virtual mailbox delivery. This bypasses any .forward files that might be in my virtual mailbox dir (e.g. /home/vmail/cerberusonline.com/cweldon/.forward does nothing) and just delivers straight into the maildir. According to Tadek’s Blog, he read that using Maildrop looked to be the best solution as it can interface with the database to find the home and mailbox directories.

I came to find out that Maildrop is a subset of the Courier MTA. It’s actually supposed to replace procmail, and although it is written in C++ and is larger in size than procmail, it’s apparently more efficient at filtering and appropriately routing mail than procmail. The biggest problem I had though was there was little to no documentation on how to get procmail to lookup the user tables in Postgres and pick up the .mailfilter file appropriately.

It turns out, that if you already have courier-authlib setup and running on your system, there is nothing you need to do in order to get Maildrop to use it (you can specify the alternate -a flag to force Courier-AuthLib, but in my case it was unnecessary). The catch is you have to understand how you’ve got your lookups going. If you follow the PostFix maildrop readme, you’ll notice it states that using ${RECIPIENT} works in most cases. In my setup, I’m doing lookups based on just username, which meant I had to use the following statement in my PostFix master.cf file:

maildrop  unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${user}

I found this out the hard way when I ran the following two commands on the command line:

root # maildrop -V 7 -d cweldon@cerberusonline.com
Invalid user specified.

root # maildrop -V 7 -d cweldon
maildrop: authlib: groupid=5000
maildrop: authlib: userid=5000
maildrop: authlib: logname=cweldon, home=/home/vmail/cerberusonline.com/cweldon/, mail=/home/vmail/cerberusonline.com/cweldon/.maildir/
maildrop: Changing to /home/vmail/cerberusonline.com/cweldon/

Maildrop did the lookup via authlib, as indicated by the second entry. Using authtest, the results are mimicked:

root # authtest cweldon@cerberusonline.com
Authentication FAILED: Operation not permitted
root # authtest cweldon
Authentication succeeded.

     Authenticated: cweldon  (uid 5000, gid 5000)
    Home Directory: /home/vmail/cerberusonline.com/cweldon/
           Maildir: /home/vmail/cerberusonline.com/cweldon/.maildir/
             Quota: (none)
Encrypted Password: NOT SHOWING
Cleartext Password: (none)
           Options: (none)

So, someone is likely asking how I have my maildroprc file setup. It’s rather simplistic:

# Global maildrop filter file

#DEFAULT="$HOME/.maildir/"

# Define variables
SHELL="/bin/bash"
#DEFAULT = "$HOME/.maildir"
#MAILDIR = "$HOME/.maildir"

#
# Logfile destination
#
logfile "${HOME}/.maildrop.log"

`test -r $HOME/.mailfilter`
if( $RETURNCODE == 0)
        {
        log "(==) Including $HOME/.mailfilter"
                exception {
                        include $HOME/.mailfilter
                }
        }

What this does is use the $HOME as returned by Courier-AuthLib to check if I have a .mailfilter in my virtual home directory. If I do, it includes that before dropping my mail into my maildir.