Welcome on DoYourself.org

Have an tutorial and wanna public it ?

On this website you can add any type of tutorials , but first you must have an account . To create one just complete right inputs with your dates.

Members Login

Lost your password?

Not member yet? Sign-up!

Security Code

How do I expire a PHP session after 30 minutes?

Posted by CarcaBot on Tuesday, 11.10.09 @ 06:50am  •  Filled under PHP  • (1) Comment  •   •  Views (171)  

You should implement a session timeout on your own. Both session.gc_maxlifetime and session.cookie_lifetime are not reliable. The reason for that is:

First:

session.gc_maxlifetime
session.gc_maxlifetime specifies the number of seconds after which data will be seen as 'garbage' and cleaned up. Garbage collection occurs during session start.

But the garbage collector is only started with a probability of session.gc_probability devided by session.gc_divisor. And using the default values for that options (1 and 100), the chance is only at 1%.

Furthermore the age of the session data is calculated on the file’s last modification date and not the last access date:

Note: If you are using the default file-based session handler, your filesystem must keep track of access times (atime). Windows FAT does not so you will have to come up with another way to handle garbage collecting your session if you are stuck with a FAT filesystem or any other filesystem where atime tracking is not available. Since PHP 4.2.3 it has used mtime (modified date) instead of atime. So, you won't have problems with filesystems where atime tracking is not available.

So it additionally might occur that a session data file is deleted while the session itself is still considered as valid.

And second:

session.cookie_lifetime
session.cookie_lifetime specifies the lifetime of the cookie in seconds which is sent to the browser. […]

This does only affect the cookie lifetime. But the session itself may be still valid. It’s the server’s task to invalidate a session, not the client’s.

So the best solution would be to implement a session timeout on your own:

if (!isset($_SESSION['CREATED'])) {
    $_SESSION
['CREATED'] = time();
} else if (time() - $_SESSION['CREATED'] > 1800) {
   
// session started more than 30 minates ago
    session_destroy
();
    $_SESSION
= array();
}

You can scrap sessions after a certain lifespan by using the session.gc-maxlifetime ini setting:

ini_set('session.gc-maxlifetime', 60*30);

Manipulate Zip Files with PHP

Posted by CarcaBot on Sunday, 10.25.09 @ 08:48am  •  Filled under PHP  • No comments  •   •  Views (175)  

Back in the olden days the easiest way to create zip files using php was to have the operating system (OS) do it. Sure, there were modules you could compile php with but really affected portability. So using the OS wasn’t too big of an issue but it still wasn’t optimal.

Zip Files

Zip Files

The Old Way

Assuming the OS was Linux, and if you were doing php web development it probably was, you’d execute a call like the below to create a gzipped archive:

<?php
exec(tar cfvz ".$WhereBackup."/".$Name.".tar.gz ".$whatBackup.");
?>

The above sucks for a couple reasons. For one, it requires a pretty large security hole to not be plugged; exec allows piping commands to the OS directly. For another, the above method will only work on Linux; Windows doesn’t have a “tar” command.

The biggest reason the above sucked, for me at least, was that the layman doesn’t understand the tar.gz file extension. I can’t count the number of times I’ve had to explain to a client or colleague that tar.gz is, as far as their needs go, that tar.gz is ok. “You can open it in WinZip or WinRar.”, I’d say. Sigh…

Enter PclZip

PclZip is a php class that creates and manage ZIP formatted archives. PclZip works on both Windows and Linux and is pretty easy to use. There’s a pretty extensive user manual too.

I first heard about it while writing the WordPress plugin iTunes-Data. I wanted to allow the upload of the iTunes XML files but mine was over 10MB so testing was becoming… inconvenient. I knew WordPress could manipulate zip archives so I took a look under the hood and the was PclZip.

Anyway, you can create zip files, remove files from existing zip files and extract zip files using the PclZip class. Below are a couple examples of how to do each.

The Basics

The most important thing to know about using PclZip is that the PclZip object must be instantiated with a “.zip” file being passed. It doesn’t matter if you’re creating, extracting or modifying, you have to pass the file you want to manipulate before you do anything else.

<?php
require_once('pclzip.lib.php');
$archive = new PclZip('tmp/archive.zip');
?>

Optional Arguments

PclZip allows for detailed control over archives through the use of optional arguments that get passed at the tail end of functions. According to the official site:

The optional arguments are identified by a name, which is in reality a static integer value. The value of the argument can be a single value or a list of values. In some cases they does not take a value, their name is enought to indicate a specific action to the method.

There’s far too many arguments you can use so I won’t bother listing them all here but there’s a complete list at the end of the article.

Create a Zip File

To create a zip file you have to pass the name of the archive you want to create to the object. The file passed will be where to the archive will be created.

You can include multiple files by either passing the directory, if you want the entire directory or by using a couple different methods to pass individual files and directories; arrays or a csv string.

<?php
require_once('pclzip.lib.php');
$archive = new PclZip('tmp/archive.zip');
 
//includes the file "debug.cl" and the directory "logs"
$files = './debug.cl,./logs/';
if ($archive->create($files) == 0) {
die('Error : '.$archive->errorInfo(true));
}
?>

or

<?php
require_once('pclzip.lib.php');
$archive = new PclZip('tmp/archive.zip');
 
//same as above
$files = array('./debug.cl','./logs/');
if ($archive->create($files) == 0) {
die('Error : '.$archive->errorInfo(true));
}
?>

Extracting Files from Archive

There are quite a few options for extracting files from a zip archive. PclZip contains a pretty extensive filtering mechanism that allows for some pretty selective extractions.

By default when extracting the files from an archive PclZip puts the files relative to where the script is executed.

$archive = new PclZip('tmp/archive.zip');
if ($archive->extract() == 0) {
die("Error : ".$archive->errorInfo(true));
}

You can explicitly state where the archive files are extracted by setting a value to the “PCLZIP_OPT_PATH” parameter. The below extracts all the files to the “./tmp” directory:

$archive = new PclZip('tmp/archive.zip');
if ($archive->extract(PCLZIP_OPT_PATH, "./tmp") == 0) {
die("Error : ".$archive->errorInfo(true));
}

Optional Argument List

PCLZIP_OPT_PATH
PCLZIP_OPT_ADD_PATH
PCLZIP_OPT_REMOVE_PATH
PCLZIP_OPT_REMOVE_ALL_PATH
PCLZIP_OPT_SET_CHMOD
PCLZIP_OPT_BY_NAME
PCLZIP_OPT_BY_EREG
PCLZIP_OPT_BY_PREG
PCLZIP_OPT_BY_INDEX
PCLZIP_OPT_EXTRACT_AS_STRING
PCLZIP_OPT_EXTRACT_IN_OUTPUT
PCLZIP_OPT_NO_COMPRESSION
PCLZIP_OPT_COMMENT
PCLZIP_OPT_ADD_COMMENT
PCLZIP_OPT_PREPEND_COMMENT
PCLZIP_OPT_REPLACE_NEWER
PCLZIP_OPT_EXTRACT_DIR_RESTRICTION
PCLZIP_OPT_ADD_TEMP_FILE_ON
PCLZIP_OPT_ADD_TEMP_FILE_THRESHOLD
PCLZIP_OPT_ADD_TEMP_FILE_OFF

How to unzip ZIP files with PHP

Posted by CarcaBot on Sunday, 10.25.09 @ 08:47am  •  Filled under PHP  • No comments  •   •  Views (242)  

How to automatically unzip uploaded zip files with PHP

I wanted to upload a zip file with php script and unzip it within the server running PHP and linux. How to do it? i am documenting here about a solution to unpack uploaded zip files locally within the server with PHP.

well! it can be done in 3 ways. I

1. Using PHP exec command
2. suPHP (Alternative to PHPsuexec)
3. Get Pclzip class library. (create and extract ZIP files
4. Using Zziplib with PHP

I was successful with (3) option.

I tried the first option of using php exec command calling system level but it never worked for me and it just kept getting checkdir errors in my apache log.


  1. <?php  
  2. // uses  unzip command just like in commandline  
  3. //  
  4. //  
  5. echo system('unzip uploads/test.zip',$output);  
  6. print_r($output);  
  7.   
  8. ?> 
checkdir error: cannot create ithemex
unable to process xtheme/images/mini-archive.gif.
checkdir error: cannot create ithemex
unable to process itheme-x/images/mini-archive.png.

Using suPHP

suPHP is a good method just because instead of apache running as nobody/apache it runs under user account ownership. Basically what this means that it will have no problem with file permissions for zip files. Another great advantage of suphp is security (it restricts local users reading php files).

Unfortunately i couldnt use suphp because it needs PHP to be running as CGI, but mine was apache module. I couldnt use suPHP

Using Pclzip class for Zip files

This method worked best for me. I uploaded a zip and it cleanly unpacked within my server. Just download the pclzip class library. Set the folder permissions to 755 where you are unpacking the files. It need not be 777. If that 755 doesnt work set the ownership and usergroup to apache or nobody.

Now upload a sample zip file test.zip and put the following code in php file


  1. <?php  
  2.  require_once('pclzip.lib.php');  
  3.  $archive = new PclZip('test.zip');  
  4.  if (($v_result_list = $archive->extract(PCLZIP_OPT_PATH, 'uploads')) == 0) {  
  5.    die("Error : ".$archive->errorInfo(true));  
  6.  }  
  7.  echo "<pre>";  
  8.  var_dump($v_result_list);  
  9.  echo "</pre>";  
  10. >

PCLZIP_OPT_PATH, ‘uploads’ -> is the destination unzip folder (give it 777 or 755 permissions)

Just the php script and you will see error codes if any. In my case it worked perfectly fine unpacking the zip file. Refer to Pclzip documentation.

Note: You must have Zlib support installed for this script to work.
To install Zlib with PHP just do yum install zlib zip php-zip

zlibsupp.GIF

Dont forget to check apache logs to trace the exact error!

Search a drive for large files in Linux

Posted by CarcaBot on Wednesday, 10.14.09 @ 08:29am  •  Filled under Linux  • No comments  •   •  Views (208)  

Q. How do I find out all large files in a directory?

A. There is no single command that can be used to list all large files. But, with the help of find command and shell pipes, you can easily list all large files.

Linux List All Large Files

To finds all files over 50,000KB (50MB+) in size and display their names, along with size, use following syntax:

Syntax for RedHat / CentOS / Fedora Linux

find {/path/to/directory/} -type f -size +{size-in-kb}k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'
Search or find big files Linux (50MB) in current directory, enter:
$ find . -type f -size +50000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'
Search in my /var/log directory:
# find /var/log -type f -size +100000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

Syntax for Debian / Ubuntu Linux

find {/path/to/directory} -type f -size +{file-size-in-kb}k -exec ls -lh {} \; | awk '{ print $8 ": " $5 }'
Search in current directory:
$ find . -type f -size +10000k -exec ls -lh {} \; | awk '{ print $8 ": " $5 }'
Sample output:

./.kde/share/apps/akregator/Archive/http___blogs.msdn.com_MainFeed.aspx?Type=AllBlogs.mk4: 91M
./out/out.tar.gz: 828M
./.cache/tracker/file-meta.db: 101M
./ubuntu-8.04-desktop-i386.iso: 700M
./vivek/out/mp3/Eric: 230M

Above commands will lists files that are are greater than 10,000 kilobytes in size. To list all files in your home directory tree less than 500 bytes in size, type:
$ find $HOME -size -500b
OR
$ find ~ -size -500b

To list all files on the system whose size is exactly 20 512-byte blocks, type:
# find / -size 20

Perl hack: To display large files

Jonathan has contributed following perl code print out stars and the length of the stars show the usage of each folder / file from smallest to largest on the box:

 du -k | sort -n | perl -ne 'if ( /^(\d+)\s+(.*$)/){$l=log($1+.1);$m=int($l/log(1024)); printf  ("%6.1f\t%s\t%25s  %s\n",($1/(2**(10*$m))),(("K","M","G","T","P")[$m]),"*"x (1.5*$l),$2);}'

ls command: finding the largest files in a directory

You can also use ls command:
$ ls -lS
$ ls -lS | less
$ ls -lS | head +10

ls command: finding the smallest files in a directory

Use ls command as follows:
$ ls -lSr
$ ls -lSr | less
$ ls -lSr | tail -10

Find r57 and c99 Shells Hidden Inside PHP and TXT Files

Posted by CarcaBot on Wednesday, 10.7.09 @ 06:09am  •  Filled under Linux  • No comments  •   •  Views (345)  

When malicious intruders compromise a web server, there’s an excellent chance a famous Russian PHP script, r57shell, will follow. The r57shell PHP script gives the intruder a number of capabilities, including, but not limited to: downloading files, uploading files, creating backdoors, setting up a spam relay, forging email, bouncing a connection to decrease the risk of being caught, and even taking control of SQL databases. All these functions become readily available through an easy to use web interface, but now you can fight back.

A Turkish member on a forum I participate in released this nifty little bash command, but first, make sure you execute updatedb so find has an up to date image to search:

find /var/www/  -name "*".php  -type f -print0  | xargs -0 grep r57 | uniq -c  | sort -u  | cut -d":" -f1  | awk '{print "rm -rf " $2}' | uniq

You can also search regular text (.txt) files:

find /var/www/  -name "*".txt  -type f -print0  | xargs -0 grep r57 | uniq -c  | sort -u  | cut -d":" -f1  | awk '{print "rm -rf " $2}' | uniq

Or even cleverly disguised GIF image files:

find /var/www/  -name "*".gif  -type f -print0  | xargs -0 grep r57 | uniq -c  | sort -u  | cut -d":" -f1  | awk '{print "rm -rf " $2}' | uniq

The command might appear scary, or even malicious to an inexperienced Linux admin, but here’s the break down.

find /var/www/

find is a must know command when dealing with Linux. Find is what’s used to perform command line file searches. The path /var/www is the directory find will search, in addition to all directories contained within www, but nothing above. For example, /var/mail is not searched. If your publicly accessible files are not contained in /var/www, then you’ll obviously need to replace /var/www with the correct path.

-name "*".php  -type f -print0

This portion of the command tells find to search file names (not directories) ending in .php. Anything else is ignored.

| xargs -0 grep r57

The pipe symbol ( | ) tells Linux to take the results of the first command (the PHP files we searched for), and pass them along to the second command, xargs. At this point, all located files are searched for any mention of r57, not just the file names, but the actual content within the files.

| uniq -c  | sort -u

uniq will prevent duplicate results from displaying. The command is smart enough to know when multiple instances are found in a single file, resulting in a single mention instead of potentially hundreds, flooding your console with repeated messages. The -c parameter tells uniq to count the number of consecutive lines that were combined. sort will take the unordered results, and display them in some type of orderly fashion.

| cut -d":" -f1

cut will prevent the line of code that contains r57 from showing up in the results. The output is just a simple mention of the filename or names, and how many occurrences. There’s no need to display the actual code if your intentions are to remove the malicious files.

| awk '{print "rm -rf " $2}'

awk, a programming language in itself, is a very powerful command with many beneficial uses. In this command, awk is instructed to print rm -rf with the file path and file name appended. Here’s an example output:

rm -rf /var/www/users/domain.com/images/uploads/r57shell.php

rm -rf is used to delete files without asking questions. The, “are you sure you want to delete …” is skipped, so be careful when using the -rf switch, it’s very destructive if used without care. Notice the print portion - this means the command is only printed, not carried out. Once you’ve confirmed all the found files are malicious, you can easily dumb the results into a file, make the file executable, and delete the plague in one shot instead of manually deleting individual files one by one.

Another popular tool is the c99shell, which I also recommend searching for. Just change three characters:

find /var/www/  -name "*".php  -type f -print0  | xargs -0 grep c99 | uniq -c  | sort -u  | cut -d":" -f1  | awk '{print "rm -rf " $2}' | uniq

If you’re interested in seeing an example of the c99shell interface, here’s a rooted site:

http://www.iett.gov.tr/en/kitap/

Crontab every five minutes

Posted by CarcaBot on Friday, 09.4.09 @ 22:31pm  •  Filled under Linux  • No comments  •   •  Views (315)  

Well, i want to schedules to run one of my file every 5 minutes. Then what would be the best choose. Yes, CRON.
I can schedule my cron to run that file every five minutes to execute my desired results. Thats why i want to make a note on this regards.

To edit the crontab i use the following command:

$ crontab -e

To list my currnet crontab

$ crontab -l

The following is the format entries in a crontab must be. Note all lines starting with # are ignored, comments.

So in terminal print ‘Hello’ every 5 minutes..


# MIN HOUR MDAY MON DOW COMMAND

*/5 * * * * echo 'Hello'
MIN	Minute 	 0-60
HOUR Hour [24-hour clock] 0-23
MDAY Day of Month 1-31
MON Month 1-12 OR jan,feb,mar,apr ...
DOW Day of Week 0-6 OR
sun,mon,tue,wed,thu,fri,sat
COMMAND Command to be run Any valid command-line

Examples

Here are a few examples, to see what some entries look like.

#Run command at 7:00am each weekday [mon-fri]
00 07 * * 1-5 mail_pager.script ‘Wake Up’

#Run command on 1st of each month, at 5:30pm
30 17 1 * * pay_rent.script

#Run command at 8:00am,10:00am and 2:00pm every day
00 8,10,14 * * * do_something.script

#Run command every 5 minutes during market hours
*/5 6-13 * * mon-fri get_stock_quote.script

#Run command every 3-hours while awake
0 7-23/3 * * * drink_water.script

Special Characters in Crontab

You can use an

asterisk

in any category to mean for every item, such as every day or every month.

You can use commas in any category to specify multiple values. For example: mon,wed,fri

You can use dashes to specify ranges. For example: mon-fri, or 9-17

You can use forward slash to specify a repeating range. For example: */5 for every five minutes, hours, days
Special Entries

There are several special entries, some which are just shortcuts, that you can use instead of specifying the full cron entry.

The most useful of these is probably @reboot which allows you to run a command each time the computer gets reboot. This could be useful if you want to start up a server or daemon under a particular user, or if you do not have access to the rc.d/init.d files.

Example Usage:

# restart freevo servers
@reboot freevo webserver start
@reboot freevo recordserver start

The complete list:

Entry Description Equivalent To
@reboot Run once, at startup. None
@yearly Run once a year 0 0 1 1 *
@annually (same as @yearly) 0 0 1 1 *
@monthly Run once a month 0 0 1 * *
@weekly Run once a week 0 0 * * 0
@daily Run once a day 0 0 * * *
@midnight (same as @daily) 0 0 * * *
@hourly Run once an hour 0 * * * *

Miscelleanous Issues

Script Output
If there is any output from your script or command it will be sent to that user’s e-mail account, on that box. Using the default mailer which must be setup properly.

You can set the variable MAILTO in the crontab to specify a separate e-mail address to use. For example:
MAILTO=”admin@mydomain.com”

Redirect Output to /dev/null
You can redirect the output from a cron script to /dev/null which just throws it away. By redirecting to /dev/null you will not receive anything from the script, even if it is throwing errors.
* * * * * /script/every_minute.pl > /dev/null 2>&1

Missed Schedule Time
Cron does not run a command if it was missed. Your computer must be running for cron to run the job at the time it is scheduled. For example, if you have a 1:00am scheduled job and your computer was off at that time, it will not run the missed job in the morning when you turn it on.

Linux Crontab: 15 Awesome Cron Job Examples

Posted by CarcaBot on Friday, 09.4.09 @ 22:26pm  •  Filled under Linux  • (1) Comment  •   •  Views (220)  

Linux Crontab Guide
An experienced Linux sysadmin knows the importance of running the routine maintenance jobs in the background automatically.

Linux Cron utility is an effective way to schedule a routine background job at a specific time and/or day on an on-going basis.

This article is part of the on-going series. In this article, let us review 15 awesome examples of crontab job scheduling.

Linux Crontab Format

MIN HOUR DOM MON DOW CMD
Table: Crontab Fields and Allowed Ranges (Linux Crontab Syntax) Field Description Allowed Value
MIN Minute field 0 to 59
HOUR Hour field 0 to 23
DOM Day of Month 1-31
MON Month field 1-12
DOW Day Of Week 0-6
CMD Command Any command to be executed.

1. Scheduling a Job For a Specific Time Every Day

The basic usage of cron is to execute a job in a specific time as shown below. This will execute the Full backup shell script (full-backup) on 10th June 08:30 AM.

Please note that the time field uses 24 hours format. So, for 8 AM use 8, and for 8 PM use 20.

30 08 10 06 * /home/ramesh/full-backup
  • 30 – 30th Minute
  • 08 – 08 AM
  • 10 – 10th Day
  • 06 – 6th Month (June)
  • * – Every day of the week

2. Schedule a Job For More Than One Instance (e.g. Twice a Day)

The following script take a incremental backup twice a day every day.

This example executes the specified incremental backup shell script (incremental-backup) at 11:00 and 16:00 on every day. The comma separated value in a field specifies that the command needs to be executed in all the mentioned time.

00 11,16 * * * /home/ramesh/bin/incremental-backup
  • 00 – 0th Minute (Top of the hour)
  • 11,16 – 11 AM and 4 PM
  • * – Every day
  • * – Every month
  • * – Every day of the week

3. Schedule a Job for Specific Range of Time (e.g. Only on Weekdays)

If you wanted a job to be scheduled for every hour with in a specific range of time then use the following.

Cron Job everyday during working hours

This example checks the status of the database everyday (including weekends) during the working hours 9 a.m – 6 p.m

00 09-18 * * * /home/ramesh/bin/check-db-status
  • 00 – 0th Minute (Top of the hour)
  • 09-18 – 9 am, 10 am,11 am, 12 am, 1 pm, 2 pm, 3 pm, 4 pm, 5 pm, 6 pm
  • * – Every day
  • * – Every month
  • * – Every day of the week

Cron Job every weekday during working hours

This example checks the status of the database every weekday (i.e excluding Sat and Sun) during the working hours 9 a.m – 6 p.m.

00 09-18 * * 1-5 /home/ramesh/bin/check-db-status
  • 00 – 0th Minute (Top of the hour)
  • 09-18 – 9 am, 10 am,11 am, 12 am, 1 pm, 2 pm, 3 pm, 4 pm, 5 pm, 6 pm
  • * – Every day
  • * – Every month
  • 1-5 -Mon, Tue, Wed, Thu and Fri (Every Weekday)

4. How to View Crontab Entries?

View Current Logged-In User’s Crontab entries

To view your crontab entries type crontab -l from your unix account as shown below.

ramesh@dev-db$ crontab -l
@yearly /home/ramesh/annual-maintenance
*/10 * * * * /home/ramesh/check-disk-space

[Note: This displays crontab of the current logged in user]

View Root Crontab entries

Login as root user (su – root) and do crontab -l as shown below.

root@dev-db# crontab -l
no crontab for root

Crontab HowTo: View Other Linux User’s Crontabs entries

To view crontab entries of other Linux users, login to root and use -u {username} -l as shown below.

root@dev-db# crontab -u sathiya -l
@monthly /home/sathiya/monthly-backup
00 09-18 * * * /home/sathiya/check-db-status

5. How to Edit Crontab Entries?

Edit Current Logged-In User’s Crontab entries

To edit a crontab entries, use crontab -e as shown below. By default this will edit the current logged-in users crontab.

ramesh@dev-db$ crontab -e
@yearly /home/ramesh/centos/bin/annual-maintenance
*/10 * * * * /home/ramesh/debian/bin/check-disk-space
~
"/tmp/crontab.XXXXyjWkHw" 2L, 83C

[Note: This will open the crontab file in Vim editor for editing.
Please note cron created a temporary /tmp/crontab.XX... ]

When you save the above temporary file with :wq, it will save the crontab and display the following message indicating the crontab is successfully modified.

~
"crontab.XXXXyjWkHw" 2L, 83C written
crontab: installing new crontab

Edit Root Crontab entries

Login as root user (su – root) and do crontab -e as shown below.

root@dev-db# crontab -e

Edit Other Linux User’s Crontab File entries

To edit crontab entries of other Linux users, login to root and use -u {username} -e as shown below.

root@dev-db# crontab -u sathiya -e
@monthly /home/sathiya/fedora/bin/monthly-backup
00 09-18 * * * /home/sathiya/ubuntu/bin/check-db-status
~
~
~
"/tmp/crontab.XXXXyjWkHw" 2L, 83C

6. Schedule a Job for Every Minute Using Cron.

Ideally you may not have a requirement to schedule a job every minute. But understanding this example will will help you understand the other examples mentioned below in this article.

* * * * * CMD

The * means all the possible unit — i.e every minute of every hour through out the year. More than using this * directly, you will find it very useful in the following cases.

  • When you specify */5 in minute field means every 5 minutes.
  • When you specify 0-10/2 in minute field mean every 2 minutes in the first 10 minute.
  • Thus the above convention can be used for all the other 4 fields.

7. Schedule a Background Cron Job For Every 10 Minutes.

Use the following, if you want to check the disk space every 10 minutes.

*/10 * * * * /home/ramesh/check-disk-space

It executes the specified command check-disk-space every 10 minutes through out the year. But you may have a requirement of executing the command only during office hours or vice versa. The above examples shows how to do those things.

Instead of specifying values in the 5 fields, we can specify it using a single keyword as mentioned below.

There are special cases in which instead of the above 5 fields you can use @ followed by a keyword — such as reboot, midnight, yearly, hourly.

Table: Cron special keywords and its meaning Keyword Equivalent
@yearly 0 0 1 1 *
@daily 0 0 * * *
@hourly 0 * * * *
@reboot Run at startup.

8. Schedule a Job For First Minute of Every Year using @yearly

If you want a job to be executed on the first minute of every year, then you can use the @yearly cron keyword as shown below.

This will execute the system annual maintenance using annual-maintenance shell script at 00:00 on Jan 1st for every year.

@yearly /home/ramesh/red-hat/bin/annual-maintenance

9. Schedule a Cron Job Beginning of Every Month using @monthly

It is as similar as the @yearly as above. But executes the command monthly once using @monthly cron keyword.

This will execute the shell script tape-backup at 00:00 on 1st of every month.

@monthly /home/ramesh/suse/bin/tape-backup

10. Schedule a Background Job Every Day using @daily

Using the @daily cron keyword, this will do a daily log file cleanup using cleanup-logs shell scriptat 00:00 on every day.

@daily /home/ramesh/arch-linux/bin/cleanup-logs "day started"

11. How to Execute a Linux Command After Every Reboot using @reboot?

Using the @reboot cron keyword, this will execute the specified command once after the machine got booted every time.

@reboot CMD

12. How to Disable/Redirect the Crontab Mail Output using MAIL keyword?

By default crontab sends the job output to the user who scheduled the job. If you want to redirect the output to a specific user, add or update the MAIL variable in the crontab as shown below.

ramesh@dev-db$ crontab -l
MAIL="ramesh"

@yearly /home/ramesh/annual-maintenance
*/10 * * * * /home/ramesh/check-disk-space

[Note: Crontab of the current logged in user with MAIL variable]


If you wanted the mail not to be sent to anywhere, i.e to stop the crontab output to be emailed, add or update the MAIL variable in the crontab as shown below.

MAIL=""

13. How to Execute a Linux Cron Jobs Every Second Using Crontab.

You cannot schedule a every-second cronjob. Because in cron the minimum unit you can specify is minute. In a typical scenario, there is no reason for most of us to run any job every second in the system.

14. Specify PATH Variable in the Crontab

All the above examples we specified absolute path of the Linux command or the shell-script that needs to be executed.

For example, instead of specifying /home/ramesh/tape-backup, if you want to just specify tape-backup, then add the path /home/ramesh to the PATH variable in the crontab as shown below.

ramesh@dev-db$ crontab -l

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/home/ramesh

@yearly annual-maintenance
*/10 * * * * check-disk-space

[Note: Crontab of the current logged in user with PATH variable]

15. Installing Crontab From a Cron File

Instead of directly editing the crontab file, you can also add all the entries to a cron-file first. Once you have all thoese entries in the file, you can upload or install them to the cron as shown below.

ramesh@dev-db$ crontab -l
no crontab for ramesh

$ cat cron-file.txt
@yearly /home/ramesh/annual-maintenance
*/10 * * * * /home/ramesh/check-disk-space

ramesh@dev-db$ crontab cron-file.txt

ramesh@dev-db$ crontab -l
@yearly /home/ramesh/annual-maintenance
*/10 * * * * /home/ramesh/check-disk-space

Note: This will install the cron-file.txt to your crontab, which will also remove your old cron entries. So, please be careful while uploading cron entries from a cron-file.txt.

mysql query - out of memory problem

Posted by racakg on Tuesday, 09.1.09 @ 19:01pm  •  Filled under MySQL  • (1) Comment  •   •  Views (300)  

Hi,

Every time I access 'admin/logs/referrers' page I have this error:

Warning: mysql_query() [function.mysql-query]: Unable to save result set in /home/morazzia/public_html/includes/database.mysql.inc on line 102

Warning: Out of memory (Needed 1048506 bytes) query: pager_query SELECT COUNT(DISTINCT(url)) FROM accesslog WHERE url <> '' AND url NOT LIKE '%www.morazzia.com%' in /home/morazzia/public_html/includes/database.mysql.inc on line 121

I am on dedicated server running dual core 3.4Mhz with 2gb of memory. I have 10k visitors and 120.000 page views per day. I enable cash support on drupal with cron job (every 4 hours). I enable deval module. This is first couple of lines in deval log for referrers page:

Executed 122 queries in 100.85 milliseconds. Queries taking longer than 5 ms and queries executed more than once, are highlighted.Page execution time was 1352.91 ms.ms # where query
63.67 1 pager_query SELECT url, COUNT(url) AS hits, MAX(timestamp) AS last FROM accesslog WHERE url NOT LIKE '%www.morazzia.com%' AND url <> '' GROUP BY url ORDER BY hits DESC LIMIT 0, 30
1.08 1 forum_block SELECT n.nid, n.title, l.comment_count FROM node n INNER JOIN node_comment_statistics l ON n.nid = l.nid WHERE n.type = 'forum' AND n.status = 1 ORDER BY n.nid DESC LIMIT 0, 5
0.92 1 user_block SELECT COUNT(sid) AS count FROM sessions WHERE timestamp >= 1167293169 AND uid = 0
0.89 1 module_list SELECT name, filename, throttle, bootstrap FROM system WHERE type = 'module' AND status = 1 ORDER BY weight ASC, filename ASC
0.87 1 drupal_lookup_path SELECT dst FROM url_alias WHERE src = 'http://www.baberoad.com/'
0.65 1 cache_get SELECT data, created, headers, expire FROM cache WHERE cid = 'menu:1:en'
0.63 1 pager_query SELECT COUNT(DISTINCT(url)) FROM accesslog WHERE url <> '' AND url NOT LIKE '%www.morazzia.com%'
0.58 1 block_list SELECT * FROM blocks WHERE theme = 'morazzia' AND status = 1 ORDER BY region, weight, module

Here you can see my php info page: http://www.morazzia.com/phpinfo.php

Any advice?

Regards,
Rale

Linux - Find string in files.

Posted by CarcaBot on Sunday, 08.30.09 @ 12:29pm  •  Filled under Linux  • No comments  •   •  Views (200)  

Linux - Find string in files.

find . | xargs grep 'string' -sl

The -s is for summary and won't display warning messages such as grep: ./directory-name: Is a directory

The -l is for list, so we get just the filename and not all instances of the match displayed in the results.

Performing the search on the current directory I get:


./javascript_open_new_window_form.php
./excel_large_number_error.php
./linux_vi_string_substitution.php
./email_reformat.php
./online_email_reformat.php
./excel_find_question_mark.php
./linux_find_string_in_files.php
./excel_keyboard_shortcuts.php
./linux_grep.php
./md5_unique_sub_string.php
./email_reformat_token.php
./excel_password_protect.php
./mysql_date_calulation.php
./md5_string.php
./php_javascript_passing_values_to_new_window_in_url.php
./php_math_on_string/math_on_string_form.php
./guide.php
./excel_large_number_paste.php
./piping_commands_find_grep_sed.php
./google-search-for-seo-research.php
./filename_conversion_form.php
./linux_find_string_files.php

I find this useful for just quickly seeing which files contain a search time. I would normally limit the files searched with a command such as :

find . -iname '*php' | xargs grep 'string' -sl

Another common search for me, is to just look at the recently updated files:

find . -iname '*php' -mtime -1 | xargs grep 'string' -sl

would find only files edited today, whilst the following finds the files older than today:

find . -iname '*php' -mtime +1 | xargs grep 'string' -sl

YouTube, FFMPEG, and MP3 Conversion

Posted by CarcaBot on Sunday, 08.23.09 @ 16:50pm  •  Filled under FFMPEG  • No comments  •   •  Views (197)  

Yesterday I published a quick post about a basic MPEG -> FLV video conversion method using FFMPEG. Today I want to share another great usage of FFMPEG: stripping a video’s audio and creating an MP3. As an added bonus, I’ll be ripping the audio from a great YouTube video using youtube-dl, a python script with loads of options for downloading YouTube videos.

The youtube-dl Shell Script

01.#command
02.davidwalsh83$ youtube-dl.py http://www.youtube.com/watch?v=tlWpnLdPwvk
03. 
04.#output
05.[youtube] Setting language
06.[youtube] tlWpnLdPwvk: Downloading video webpage
07.[youtube] tlWpnLdPwvk: Extracting video information
08.[youtube] tlWpnLdPwvk: URL: http://www.youtube.com/get_video?video_id=tlWpnLdPwvk&;t=vjVQa1PpcFMZ4TkTSYXamGxmLZq-ot2l8Jx-HiNAf0I=&el=detailpage&ps=
09.[download] Destination: tlWpnLdPwvk.flv
10.[download] 100.0% of 18.78M at   53.98k/s ETA 00:00

The initial install of youtube-dl was tough but only because I’m not well-versed with the permissions side of Unix. I eventually figured it out but if you have trouble installing youtube-dl or simply want to use its advanced options, check out youtube-dl’s documentation. youtube-dl also provides an option to read a text file with a list of videos and will do a batch download. Note: youtube-dl allows you to set an output file name but it didn’t appear to be working. If you’d like a more simple method, Mark Sanborn wrote a great post on ripping YouTube videos using wget.

The FFMPEG Shell Script

1.davidwalsh83$ ffmpeg -i tlWpnLdPwvk.flv RodStewartMaggieMay.mp3

As I did with youtube-dl, I chose the most basic usage of FFMPEG. FFMPEG’s documentation provides you a list of all of the possible conversion options if you need them.

Use the video player above to listen to the result. The video I chose is a live version of Maggie May by Rod Stewart recorded at Royal Albert Hall in the Knightsbridge area of the City of Westminster, London, England. The video is from Rod’s One Night Only DVD which I HIGHLY recommend. This is the best version of Maggie May that exists (my opinion but, as you know, when is my opinion not correct?) Enjoy!