Archive

Archive for the ‘Programming’ Category

How to: Insert a new field into a dynamic array

September 7th, 2009 Chuck No comments

I’ve been busy with work – deadlines and all, so it’s been a while since my last entry. For now I’ll just share a quick tip. How to Insert a new field into a dynamic array.

If you’ve been reading the code snippets I’ve been posting thus far, you’ll notice that I am doing this thing a lot. Anyway, here’s a little background on this situation.

Let’s say you just retrieved records from your database table and stored them into an associative array named $results, now you want to insert an extra field for each row but you don’t know how to to assign them since you don’t know how to walk through the array.

Let’s say you have this array:

Array
(
    [0] => Array
        (
            [user_id] => 1
            [user_name] => Chuck
            [user_level] => 3
            [birthdate] => 1984-09-15
        )

    [1] => Array
        (
            [user_id] => 2
            [user_name] => Blah
            [user_level] => 1
            [birthdate] => 2001-01-01
        )

    [2] => Array
        (
            [user_id] => 3
            [user_name] => New user
            [user_level] => 1
            [birthdate] => 2002-10-30
        )

    [3] => Array
        (
            [user_id] => 4
            [user_name] => Random guy
            [user_level] => 1
            [birthdate] => 1992-07-21
        )

)

This means, each row has the following fields – user_id, user_name, user_level, and birthdate.
Now, let’s say we wanted to create a new field named “age” and store it in the same array. Here’s my preferred method of doing this:

foreach($results as $index=>$row){
     $results[$index]['age'] = get_age($row['birthdate']);
}

Take note that to maintain the abstraction of the code, I just called a function named get_age();. Remember that there is no such function so you’ll have to code that in to make it work. For completeness sake, here’s how I’d probably do it:

// Accepts 2 parameters, $bday and $now. $bday is of
// the 'YYYY-MM-DD' format. The result is based on the
// difference between $now and $bday converted to
// years. $now is optional. If it is not supplied, it uses
// the current time. This is not exactly accurate since I
// just indiscriminately used 365.25 days in my calculation.
// To improve, I'll have to detect leap years and just
// add 1 day to that, instead of 1/4 days to every year.
function get_age($bday, $now = NULL){
     $bday = strtotime($bday);
     $now = ($now)?strtotime($now):mktime();
     return floor(($now - $bday)/(60*60*24*365.25));
}

Anyway, after doing all that, you will get this result:

Array
(
    [0] => Array
        (
            [user_id] => 1
            [user_name] => Chuck
            [user_level] => 3
            [birthdate] => 1984-09-15
            [age] => 24
        )

    [1] => Array
        (
            [user_id] => 2
            [user_name] => Blah
            [user_level] => 1
            [birthdate] => 2001-01-01
            [age] => 8
        )

    [2] => Array
        (
            [user_id] => 3
            [user_name] => New user
            [user_level] => 1
            [birthdate] => 2002-10-30
            [age] => 6
        )

    [3] => Array
        (
            [user_id] => 4
            [user_name] => Random guy
            [user_level] => 1
            [birthdate] => 1992-07-21
            [age] => 17
        )

)

As an aside, you could probably use array_walk() to achieve something similar, but this method is way simpler. I hope this has been helpful to you!

Categories: PHP and MySQL Tags: , , , ,

Juggling time zones

September 3rd, 2009 Chuck 11 comments

Handling times for different timezones is one of the usual things I had to face with every project that has in one way or the other involves the element of time.

The usual projects I work on store dates into a datetime field in the database. This means that the date must use the YYYY-MM-DD HH:MM:SS format (e.g. 2009-09-03 07:45:00).

One of the first things you’d want to do is to let the user specify a timezone setting for that user account (assuming this is in a multi-user environment, as most web applications are). You could view all the timezones using this code:

echo '<ul>';
foreach(DateTimeZone::listAbbreviations() AS $timezone_abbreviation=>$timezones){
     foreach($timezones AS $timezone)
          if(isset($timezone['timezone_id']))
          echo '<li>'.$timezone_abbreviation.' - ' . $timezone['timezone_id'] . '
          <ul>
               <li>Uses DST: '
. ($timezone['dst']===TRUE?'Yes':'No') . '</li>
               <li>Offset: '
. $timezone['offset'] . '</li>
          </ul></li>'
;
}
echo '</ul>';

You could modify the code above to create a drop-down list of the different timezones to make it easier for the user to pick a timezone (because simply writing GMT+8 or UTC+8 might be confusing for some people).

Next, we will have to convert all of our stored dates into a single timezone, for simplicity, I use GMT/UTC as a reference timezone (since it is easy to add timezone offsets to it). If you store your dates in a datetime field, you can perform the following conversion after retrieving the timestamp from the database:

// let's assume $parsedtime is the entry we have in the database
// and $offset is the timezone offset we got based on the user's settings
$parsedtime = '2009-09-15 12:00:00';
$offset = 8*60*60; // this is GMT +8
// get the GMT counterpart of the parsedtime
$unix_time = strtotime($parsedtime . ' GMT');
echo gmdate('m d Y h:iA',$unix_time-$offset); // displays: 09 15 2009 04:00AM

So that means, 12nn of September 15, 2009 is 4am of September 15, 2009 in GMT/UTC.

Let’s say we had a Calendar application, and you wanted to set an alarm for 6pm on that same day. You could do the something like the this (assuming you wrote some sort of class for this thing):

$reminder = new Reminder('2009-09-15 18:00:00','Dinner with the family');
echo $reminder->timeLeft();

Since the current date and time was 12nn of September 15, 2009, this means that the above reminder will display “6 hours left”, depending on how the timeLeft() method is formatted.

If we didn’t convert the date into a standard timezone, this will force our functions to use the server’s timezone setting and that might not be what you want.

Regular expressions in MySQL

September 1st, 2009 Chuck No comments

MySQL I’ve been using MySQL for the better part of the past 6 years so it comes as a pleasant surprise when I found out last month that it can do regular expressions within its SQL!

I’m not sure about the other databases, the last time I’ve used Oracle was around 2004-2005 during my college days, so I can’t really say if this is a MySQL-exclusive feature or something along those lines. I’m pretty positive that this isn’t standard SQL though, since we never had such lessons back in college.

Anyway, if you’re familiar with SQL and string matching within SQL, it makes use of the LIKE = '[string]' clause. Where [string] is the string to be searched for. Optionally, you could also use wildcards like _ (underscore), % (percent), [charlist] (character list) or [^charlist] (negated character list) or a combination of those to match string fragments. Unfortunately this has also been one of the major points of entry for hacking/hijacking a database-driven website, via SQL injection. Due to this vulnerability, I’ve either been using heavy data validation, or store encoded data, or at times, avoiding this altogether… but I digress.

Using regex in my SQL queries is a godsend. It helps reduce my data processing and validation overhead.

MySQL supports the use of almost all POSIX regex  metacharacters via the REGEXP or RLIKE clause (or a negation using NOT REGEXP or NOT RLIKE).

The following list describes some characteristics of extended regular expressions:

  • “.” matches any single character.
  • A character class “[...]” matches any character within the brackets. For example, “[abc]” matches “a”, “b”, or “c”. To name a range of characters, use a dash. “[a-z]” matches any letter, whereas “[0-9]” matches any digit.
  • “*” matches zero or more instances of the thing preceding it. For example, “x*” matches any number of “x” characters, “[0-9]*” matches any number of digits, and “.*” matches any number of anything.
  • A REGEXP pattern match succeeds if the pattern matches anywhere in the value being tested. (This differs from a LIKE pattern match, which succeeds only if the pattern matches the entire value.)
  • To anchor a pattern so that it must match the beginning or end of the value being tested, use “^” at the beginning or “$” at the end of the pattern.

Let’s start with the traditional SELECT ...  LIKE query. Here’s a query that selects all animals whose name begins with “ant”

SELECT name

FROM animals

WHERE name LIKE 'ant%'

The results could be something like this:

name

---------

ant

anteater

antelope

...

A regex version of that would be:

SELECT name

FROM animals

WHERE name REGEXP '^ant'

So far they look similar… but, say, what if you wanted to use a complex rule like: “select all animals whose names starts with either ‘a’ or ‘c’ and ends with either ‘t’ or ‘p’.  It would look messy if you do it this way:

SELECT name

FROM animals

WHERE name LIKE 'a%t'

OR name LIKE 'a%p'

OR name LIKE 'c%t'

OR name LIKE 'c%p'

However with regex, it is as simple as this:

SELECT name

FROM animals

WHERE name REGEXP '^[ac].*[tp]$'

Imagine if your filtering conditions were much more complex. It’s not hard to see how regexp can help with that!

A neat and clean CSS menu

August 28th, 2009 Chuck 5 comments

Earlier this week, our team of developers was temporarily reassigned to help with another team’s debugging tasks due to a pressing deadline. Our front-end team was busy and only some of us from the back-end team were free to help.

The framework was based on Magento. Since our teams were all CodeIgniter developers, we found it hard to adapt to that framework. We were shocked to find out that majority of the fixes were front-end related, requiring mostly CSS and JS to fix. This was because navigating through Magento’s files was a pain. In addition, most of my teammates and colleagues from the other teams avoided the CSS bits because they either had limited CSS knowledge or were simply not confident enough to manipulate that website’s huge CSS file. So I took that task.

The website we were working on had a 3-tier navigation menu on top, controlled by JavaScript. I spent the better part of the day on that navigation bar building the content for the other menu and tackling the JS browser incompatibility problems… I previously mentioned in another entry that I suck at JS, needless to say what I did was a quick and dirty hack that works but one I’m not proud of, haha.

Anyway, 2 days later (that’s today), I was thinking – I bet it’s possible to mimic that navigation bar using purely CSS and no JS. So I set out and tried my hand at it. I relied heavily on nested  declarations in order to keep the styles neat and the markup less cluttered. Here’s the stylesheet I came up with:

html, body {
     margin: 0px;
     padding: 0px;
}
#navigation {
     width: auto;
     border: 1px solid #888;
     background-color: #EEE;
     color: #444;
     list-style: none;
     height: 22px;
     margin: 0px;
     padding: 0px;
     font-family: "Trebuchet MS";
     margin-bottom: 30px;
}
#navigation>li {
     float: left;
     width: 100px;
     text-align: center;
     height: 100%;
     cursor: pointer;
}
#navigation li:hover, #navigation>li:hover>a {
     background-color: #000;
     color: #FFF;
}
#navigation li a, #navigation li ul {
     text-decoration: none;
     color: black;
}
#navigation li a:first-letter {
     color: blue;
     font-size: 100px;
}

#navigation li ul {
     border-bottom: 1px solid black;
     border-top: 1px solid black;
     width: 100%;
     background-color: white;
     height: auto;
     display: none;
     position: absolute;
     float: left;
     left: 0px;
     list-style: none;
     overflow: hidden;
     padding: 0px;
}
#navigation li ul li {
     float: left;
     margin: 0px;
     clear: right;
     width: 200px;
     padding-left: 1em;
     padding-right: 1em;
}
#navigation li:hover ul {
     display: block;
}
#navigation li ul li:hover {
     background-color: red;
}

As for the menu’s markup itself, it consists of a master <ul> with an id of “navigation”, with nested <ul>s contained inside of it.

Here’s what the markup looks like:

<ul id="navigation">
<li><a href="#">Dogs</a>
     <ul>
          <li>Dog 1</li>
          <li>Dog 2</li>
          <li>Dog 3</li>
     </ul>
</li>
<li>Cats
     <ul>
          <li>Cat 1</li>
          <li>Cat 2</li>
          <li>Cat 3</li>
          <li>Cat 4</li>
          <li>Cat 5</li>
     </ul>
</li>
<li>Pigs</li>
</ul>
<div>Place your content here</div>

The markup looks really neat and simple with none of those pesky class="" attributes to get in the way. Also you can add as much menu items as you please.

So far this CSS declaration only supports 2-tier menus. I could try working on 3-tier, or even maybe a dynamic-depth CSS menu, if I don’t feel lazy :p.

You could see it in action by viewing this page.

Alternately, if you’re not a fan of horizontal menus, you can easily convert it into a vertical menu by removing the float: left; property in the #navigation li ul li declaration. Getting rid of the white background should be easy.

Update: August 29, 2009

I wrote another CSS declaration for 3-tier menus in vertical format. You could see the sample by going here.

Sorting uneven 2-dimensional arrays

August 27th, 2009 Chuck 1 comment

I was working on a search function for the module I was assigned to. The contents stored in the database isn’t plaintext, but rather encoded in base64. There’s currently no native MySQL functions to handle base64 data so that means I can’t make inline text and regex comparisons.

I thought hard about it and came up with 2 solutions:

  1. Save the data in plaintext instead and use heavy data validation
  2. Retrieve all rows and store them in a PHP array and do my regex magic there

Option 1 was out of the question, because in every part of the module, we’ve made sure to protect against XSS (cross site scripting). Going with this option will always risk creating a security hole regardless of how carefully I validate the supplied data.

Option 2 would be more secure and would allow me to apply my regex skills to the fullest extent, but it would cost more memory overhead because I’d have to retrieve the rows, store them into memory while doing the pattern matching…

Until I thought of option 3 (well, more like option 2.1). What I’d do is to limit the fetched rows by imposing more conditions, say for example, retrieve the rows that the currently logged user is only allowed to see. Then perform my regex thingamabob on those filtered results.

I had another problem though, doing this means I have to work on some sort of system to weigh the relevance of the results. Since this relevance value is computed after the database retrieval, this means I’ll be adding it into an extra column in the result array. The result array I am supposed to work on is a multidimensional array. It would have a similar structure as this example:

$data = array(
     array(
          'name'=>'Chuck',
          'color'=>'Blue'
     ),
     array(
          'name'=>'Rupert',
          'color'=>'Black'
     ),
     array(
          'name'=>'Cerrillo',
          'color'=>'Red'
     ),
     array(
          'name'=>'Abarro',
          'color'=>'White'
     )
);

The above array is a 2-dimensional array.  This means that echo $data[0]['name']; will display “Chuck”.

Anyway, as I earlier mentioned, I am supposed to add a relevance value to each row (or what I’d like to call weight). In addition, I might add some custom fields to some rows. Here’s how that array would look like with the additional fields and arbitrary weight values. (take note of the custom field added in line #6)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
$data = array(
     array(
          'weight'=>20,
          'name'=>'Chuck',
          'color'=>'Blue',
          'custom'=>'Boo!'
     ),
     array(
          'weight'=>10,
          'name'=>'Rupert',
          'color'=>'Black'
     ),
     array(
          'weight'=>40,
          'name'=>'Cerrillo',
          'color'=>'Red'
     ),
     array(
          'weight'=>30,
          'name'=>'Abarro',
          'color'=>'White'
     )
);

By my weight system, the “heavier” a row is, the more relevant it is to the search query. So I should sort it by weight in a descending order.

PHP has a native array function called array_multisort();, so I tried that one. It doesn’t work with my array because the array sizes are not the same. I gave up a while after looking for workarounds to get it to work with that function, so I decided to write my own sorting function for arrays like these.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
/*
* 2-Dimensional Array Sort by Chuck Cerrillo (August 26, 2009)
* Sorts a 2-dimensional array according to a given field/index.
* This is particularly useful for sorting post-processed MySQL data (i.e. where
* custom fields have been added AFTER the retrieval of table data, and you'd need
* to resort the results).
*
* For more information on how to use this array, please visit my blog entry at
* http:///blog.chuckcerrillo.com/2009/08/sorting-uneven-2-dimensional-arrays/
*/

function array_2dsort($array, $field, $sort = 'asc'){
     $temp = array();
     
     // First we grab the index names of the sub-array
     // by walking through the array
     if(is_array($array)){
          foreach($array AS $subarrayindex=>$subarray){
               if(is_array($subarray))
               foreach($subarray AS $index=>$key){
                    // Store the fieldnames into the $fieldnames array
                    $fieldnames[$index][] = $subarray[$index];
                   
                    // We then store the field names into a temporary array
                    ${$index}[$subarrayindex] = $key;
               }
          }
         
          // Sort the temporary array based according to the $sort
          // argument, and maintain index associations by using
          // asort() or arsort()
          if(strtolower($sort) == 'asc')
               asort($$field);
          else
               arsort($$field);
         
          // Now we recreate a new array with the desired order
          foreach($fieldnames AS $fieldname=>$fieldindex){
               foreach($fieldindex AS $rowindex=>$rowvalue){
                    foreach(${$field} AS $index=>$value){
                         $result[$index][$fieldname] = $array[$index][$fieldname];
                    }
               }
          }
     }
     return $result;
}

Now this one works perfectly according to my needs. This function takes in 3 parameters,

  • $array – the array to sort (required)
  • $field – the field or index to sort by (required)
  • $sort – the sorting order, either asc or desc, defaults to asc (optional)

Given the above array, I would call my function this way – $data = array_2dsort($data,'weight');. If you would do a print_r($data);, it would give you this: (note that I didn’t specify ‘desc’ as my sorting order)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
Array
(
    [1] => Array
        (
            [weight] => 10
            [name] => Rupert
            [color] => Black
            [custom] =>
        )

    [0] => Array
        (
            [weight] => 20
            [name] => Chuck
            [color] => Blue
            [custom] => Boo!
        )

    [3] => Array
        (
            [weight] => 40
            [name] => Abarro
            [color] => White
            [custom] =>
        )

    [2] => Array
        (
            [weight] => 50
            [name] => Cerrillo
            [color] => Red
            [custom] =>
        )

)

This function automagically adds blank fields to rows that do not contain them.

Regex saves the day, again

August 26th, 2009 Chuck No comments

I was supposed to do a “search” feature and a “related items” feature for the module I was assigned to. This was were my regex skills came in handy yet again.

I made a very barebone prototype of it, which takes some input text for the search query against some stored content, in this case, a full-text article (although the final version should be against a database).

The prototype I made is still very rough around the edges. What it does is that it extracts all keywords from the search query using this expression:

$regex = '/\b([\p{L}|\p{Ll}|\p{Lu}|_]+?)\b/i';

That rule can be translated as follows:

take any combination of unicode characters and underscores that are enclosed in word boundaries

Having done that, I place all matched data into an array and call that my “keywords” array, to be used later.

The next thing I did was to chop the article down into smaller pieces, currently by sentence (I only used periods as a delimiter, I should probably include other punctuation marks I guess). Then I ran preg_match() through each piece to quickly check for matches for any of the keywords. These results are then compiled.

In each compiled piece, I assign a corresponding weight. This weight is my arbitrary way of picking the best match. My currently implementation sets weight to be equal to the number of characters in the piece that are matched with keywords. I still have to refine these rules later on.

When all pieces have their weight assigned, I sort them according to weight in descending order (highest to lowest weight). Then I display the results and highlight the matched characters.

Here’s the prototype I made: search tool.

I’m still thinking of the refinements I could make so I’ll just post them here as I go along.

Also, it’s good to know that MySQL supports regex in your SQL queries, this should save me a lot of time!

Picking a JavaScript framework

August 25th, 2009 Chuck 2 comments

I had my own little experiences wrestling with JavaScript early into my webdevelopment days, but I never really went beyond simple DOM manipulation. JavaScript has always been overwhelming for me especially because I’d have to write different versions of the code as well as workarounds to ensure compatibility with most browsers… and I didn’t like rewriting stuff when I had to consider yet another browser.

Now,  a few years later, here I am again trying to get my feet wet with JS but at least there are several frameworks readily available to guide me through it, and take care of my problems (a pity they can’t help with personal problems, heh). I’m trying to choose from among the popular JS frameworks – Prototype, Scriptaculous, jQuery and Mootools. Based on my limited research, many devs prefer Mootools over the others, so I am thinking of starting with that.

So let me just formally state this before I begin…

Mootools, I choose you!