| Joined: Dec 2002 Posts: 3,255 Likes: 3 UGN Elite | UGN Elite Joined: Dec 2002 Posts: 3,255 Likes: 3 | Sup peoples. I promised another tut on PHP5 caching and I am actually going to do it. If you are like me, you have spent hours shaving precious Milli seconds off queries trying to speed up applications. Maybe you have that one query that hits 10 tables across 3 databases. Damn shame. Meanwhile your applications gets bogged down. Your users complain the site has slowed down since the new features. You know you want to keep the application but the time is killing you. Or maybe your hoster has said your application take up too many resources. I first looked into caching when Gizmo mentioned it to me. About a year maybe two years ago, we were yucking it up on the tele. I thought, caching... Ole red beard is drinking again. Caching is for desktops. Sigh, seems I was wrong. While working at the Census Department I coded quite a few LARGE data apps. There are over 300,000,000 people living in the US. So just imagine database table with more rows than 300,000,000 rows. Now we would partition the tables, that helped a lot. We had one kick [censored] DBA. But in the end, I still was lacking some speed. Enter caching If you have data you know you can count on being the same for 15 minutes or more in a high use environment... Caching is for you. When I started thinking seriously about caching I layed out some requirements. - The dataset being cached had to be secure (no web surfer should be able to directory surf and get them.)
- The file names had to be unique enough That I could reasonably be sure now accidental overwriting would happen
- The structure of the save data would have to be easy to work with. (I didn't want to write whole new code just to read and work with cache files)
- I had to be able to code this thing one time and use it for all apps.
- It had to document the cache files it created.
- It had to be flexible and allow me to change a few things.
- File extensions it uses for the cache files
- The hashing algarythm it uses, Not all systems use SHA1
- The time frame a cache is good for.
So with my requirements I set off to code a SQL (or array caching system.) | | |
▼ Sponsored Links ▼
▲ Sponsored Links ▲
| | | Joined: Dec 2002 Posts: 3,255 Likes: 3 UGN Elite | UGN Elite Joined: Dec 2002 Posts: 3,255 Likes: 3 | Below is what your buddy §intå× came up with. As always it is heavily documented. I will cover its uses below.
<?php
class cacheMgr{
public $hashType;
public $expierTime;
public $secure;
public $fileExt;
public $cachePath;
public function __construct($hashType = 1, $expierTime = 3600, $secure, $fileExt = 'tmp', $cachePath){
$this->hashType = $hashType;
$this->expierTime = (int)$expierTime;
$this->secure = (bool)$secure;
$this->fileExt = (string)$fileExt;
$this->cachePath = (string)$cachePath;
}
public function __destruct(){
unset($this->hashType);
unset($this->expierTime);
unset($this->secure);
unset($this->fileExt);
unset($this->cachePath);
}
private function getHash($plainText){
switch($this->hashType){
case 1 || 'md5':
default:
$newHash = md5($plainText, false);
break;
case 2 || 'sha1':
$newHash = sha1($plainText, false);
break;
}
return $newHash;
}
public function cacheOutArray($array, $uniquieId){
$uniquieId = str_replace(' ', '_', $uniquieId);
if(is_array($array)){
$outString = "\$".$uniquieId." = '".json_encode($array)."';";
}
if($fileName = $this->setCacheFile($uniquieId,$outString)){
$retVal = $fileName;
}else{
$retVal = false;
}
return $retVal;
}
public function getCache($fileName){
$filePath = $this->cachePath;
if(file_exists($fileName)){
@include_once($filePath.$fileName);
return true;
}else{
return false;
}
}
public function getArrayCache($uniquieId){
$uniquieId = str_replace(' ', '_', $uniquieId);
$fileName = $this->getHash($uniquieId).'.'.$this->fileExt;
$filePath = $this->cachePath;
if(file_exists($filePath.$fileName)){
include_once($filePath.$fileName);
$arr = json_decode( $$uniquieId, true );
return $arr;
}else{
return false;
}
}
private function getCacheDocBlock(){
$tmStmp = date('F d, Y - H:i:s T');
$expires = date('F d, Y - H:i:s T', mktime(date('H'), date('i'), date('s'), date('m'), date('d')+1, date('Y')));
$comment = "\n/** \n* Cache file created by cacheMgr class \n* \n* You should not edit this file unless you know what you are doing. \n* If you do edit the file and the ads system crashes, just delete \n* this file. It will be re-built.\n* @version $tmStmp \n* @expires $expires \n* @author CAMS::cacheMgr \n*/";
return $comment;
}
private function setCacheFile($uniquieId, $outString){
$uniquieId = str_replace(' ', '_', $uniquieId);
$fileName = $this->getHash($uniquieId) . '.' . $this->fileExt;
$outString = $this->prepStrForOut($outString);
$cacheFile = $this->cachePath . $fileName;
if(file_put_contents($cacheFile, $outString) === false){
return false;
}
return $fileName;
}
private function prepStrForOut($outString){
if($this->secure){
$outHeader = "<"."?php ".$this->getCacheDocBlock()." \ndefined( '_VALID_AMS_' ) or die( 'Restricted access' ); \n";
}else{
$outHeader = "<?php
".$this->getCacheDocBlock()." \n";
}
$outFooter = " \n?".">";
$outString = $outHeader . $outString . $outFooter;
return $outString;
}
}
?>
| | | | Joined: Dec 2002 Posts: 3,255 Likes: 3 UGN Elite | UGN Elite Joined: Dec 2002 Posts: 3,255 Likes: 3 | Ok, to use this you have to re-think how you use MySQL. This isn't a bad thing. You probably code in a procedural style and thats fine. But you probably treat SQL statements like strings, thats not so fine. SQL is a language that can do things to your database that will make baby Jesus cry. It is time to treat SQL like what it is. An object in your code. Enter prepared statements. http://www.php.net/manual/en/mysqli-stmt.prepare.phpNow you can use mysqli_stmt as a procedural thing but if you are changing, I suggest you use it in an OOP way. What I do now in building my apps is place all SQL into an array writen as prepared statements. When I need them I prepare them and slam the prepared statements into an array for use. The [censored] did he call me you say.... Nah, it isn't as hard as it sounds. Below is a short example of a function that might store an array of prepared statement ready queries.
function queries($key){
$queries = array(
'quer1' => 'SELECT * FROM `tableA` WHERE `id` = ?',
'quer2' => 'SELECT * FROM `tableb` WHERE `id` = ?,
'quer3' => 'SELECT
`tableb`.`col1`,
`tableb`.`col2`,
`tableb`.`col3`,
`tablea`.`col1`,
`tablea`.`col2`
FROM
`tablea`,
`tableb`
WHERE
`tablea`.`id` = ?
AND
`tableb`.`id` = ?'
);
if(!array_key_exists($key, $queries)){
$retVal = false;
}else{
$retVal = $queries[$key];
}
}
Now below is how we might use this.
function getTableA_Data($id){
$key = 'quer1';
if($query = queries($key)){
$stmt->prepare($query);
}else{
}
$stmt->bind_param("i", $id);
$stmt->execute();
$stmt->bind_result($col1, $col2, $id);
WHILE($stmt->fetch()){
$rows[] = array(
'col1' => $col1,
'col2' => $col2,
'id' => $id
);
}
$stmt->close();
return $rows;
}
Now if I wanted a data set out of tablea in the rest of me code I just do this
$tableaRows = getTableA_Data(3);
By the way I would never put all that in a function in real world code. I have centralized the handling of all prepared statements in my own wrapper of sorts. I needed to touch on this to make the caching work. So as it stands. the queries are stored in an array in a function and are served up [censored] needed. Each query is handled by its own function. But how do we keep from executing the query every time?
Last edited by §intå×; 05/31/08 02:54 AM.
| | | | Joined: Dec 2002 Posts: 3,255 Likes: 3 UGN Elite | UGN Elite Joined: Dec 2002 Posts: 3,255 Likes: 3 | The answer is simple. First we instantiate our class.
$cache = new cache( 2, 3600, true, 'php', '/path/to/cache/directory' );
| | | | Joined: Dec 2002 Posts: 3,255 Likes: 3 UGN Elite | UGN Elite Joined: Dec 2002 Posts: 3,255 Likes: 3 | Then we pass it into our functions that will do our queries. THis would be a great use of the registry pattern I posted yesterday.
$reg = new registry();
$cache = new cache( 2, 3600, true, 'php', '/path/to/cache/directory' );
$reg->set('cache',$cache);
function getTableA_Data($id, $reg){
if(!$rows = $reg->get('cache')->getArrayCache('quer1'.$id)){
if($query = queries('quer1')){
$stmt->prepare($query);
}else{
}
$stmt->bind_param("i", $id);
$stmt->execute();
$stmt->bind_result($col1, $col2, $id);
WHILE($stmt->fetch()){
$rows[] = array(
'col1' => $col1,
'col2' => $col2,
'id' => $id
);
}
$stmt->close();
$cache->cacheOutArray($rows, 'quer1'.$id);
$rows = $cache->getArrayCache('quer1'.$id);
}
return $rows;
}
So what just happened here? Well when we first enter the functions we call on the cache class method getArrayCache() to see if out cache file exists. If it does and the cache timer has not expired we return the results. If not we process the query and overwrite any cache file with the same name. The file names are created using the query keys and the parameter values we use in our queries. These are concatenated together and then hashed using either MD5 or SHA1. Okay you drunks. You have code and examples. Hit me with any and all questions. If you improve it, I am interested in what you did. Please post your modified code. | | | | Joined: Dec 2002 Posts: 3,255 Likes: 3 UGN Elite | UGN Elite Joined: Dec 2002 Posts: 3,255 Likes: 3 | Requirements of this class.
define('_VALID_AMS_', 1);
You can change '_VALID_AMS_' but you need to find it in my class and change it there too. The above examples are how I use it and find it useful. Please feel free to experiment with it. The code is released under lgpl to use and learn with. It is hoped it will be useful but no guarantees. Use at your own risk. Please throughly test before using in a production environment.
Last edited by §intå×; 05/31/08 03:33 AM.
| | | | Joined: Feb 2002 Posts: 7,203 Likes: 11 Community Owner | Community Owner Joined: Feb 2002 Posts: 7,203 Likes: 11 | For some reason I still feel like i failed... well, in teaching you how to format your code... you have so many un-needed tabs... i didnt' even look for double+ spaces... bad bad bad... | | | | Joined: Dec 2002 Posts: 3,255 Likes: 3 UGN Elite | UGN Elite Joined: Dec 2002 Posts: 3,255 Likes: 3 | Much of my code was altered on posting, but it is well formatted here on my desk top. The class itself was coded a bit ago and should look fine to you. Some of my examples are just spaces and you BBS deciding how many. Can't use tab in this post box. In my haste to get this out I forgot to post an example of what a cache file might look like.
<?php
defined( '_VALID_AMS_' ) or die( 'Restricted access' );
$_core_GetPkgNames = '["default","tkt","dorxad"]';
From "defined( '_VALID_AMS_' ) or die( 'Restricted access' ); " up to the top, the cache script generates that. You can alter the comments in the head of the document by editiing the private method "getCacheDocBlock()" in my class. Also if you wanted to edit the constant name inserted into the cache files edit the "prepStrForOut()" method. getCacheDocBlock() -line # 218 prepStrForOut($outString) - line 257 Line numbers are no good if you edit the file first. Public methods you can call in your code are - getArrayCache($uniquieId)
- getCache($fileName)
- cacheOutArray($array, $uniquieId)
I hope you finde my code decently commented and useful. But I offer no guarantees, use at your own risk. | | | | Joined: Dec 2002 Posts: 3,255 Likes: 3 UGN Elite | UGN Elite Joined: Dec 2002 Posts: 3,255 Likes: 3 | I do not see where I mentioned this above.
$cache->cacheOutArray($rows, 'quer1'.$id);
$rows = $cache->getArrayCache('quer1'.$id);
In the first method call "$cache->cacheOutArray($rows, 'quer1'.$id)" '$rows' is the array I am caching and 'quer1'.$id is the name of the query and the value of the parameter I will use in the query. "'quer1'.$id" in my class will be processed into either an md5 or sha1. This hash is then used for a file name of the cache file. This helps ensure the file names will be unique. It will also allow for multiple parameter values to be used so that any query may be cached out. You can have multiple cache object in your code. Maybe you want to have a cache lasting 15 minutes, a cache lasting 1 hrs, and one lasting a day.
$cache15min = new cache( 2, 900, true, 'php', '/path/to/cache/directory' );
$cache1hrs = new cache( 2, 3600, true, 'php', '/path/to/cache/directory' );
$cache1Day = new cache( 2, 86400, true, 'php', '/path/to/cache/directory' );
Then just used the methods I defined with the cache object you want to use.
$cache15min->cacheOutArray($membersOnline, 'memOnline'.$param.$param);
$cache1hrs->cacheOutArray($newsFeed, 'newsRssData'.$param.$param2);
$cache1day->cacheOutArray($staffArray, 'getStaffNames'.$param.$param2.$param3);
Last edited by §intå×; 05/31/08 02:48 PM.
| | |
Forums41 Topics33,840 Posts68,858 Members2,176 | Most Online3,253 Jan 13th, 2020 | | | |