Ok...What I need to do is be able to pull the raw access logs, add each entry from there into a DB, with a column for IP, Timestamp, Access Information (POST, GET, Etc), Server Code, and Bytes, running on a cron every 15 minutes.

No clue where to start...Any thoughts?

Where are Northie and Mobtex when I need them?!

Well, you would need to look up the functions like fopen() and fread().

And you would need to create a DB for it.

Not sure how much direction you need...

Curious though - why do you need it in a DB?  Is the raw log purged periodically.

MarkCCDC wrote:

Where are Northie and Mobtex when I need them?!

Working, lol

now thinking

MarkCCDC wrote:

Ok...What I need to do is be able to pull the raw access logs, add each entry from there into a DB, with a column for IP, Timestamp, Access Information (POST, GET, Etc), Server Code, and Bytes, running on a cron every 15 minutes.

No clue where to start...Any thoughts?

I'm 99.9999% sure you could buy a php/mysql script to do this for less than $100

The alternatives include

- caching your log files so that they aren't purged and downloading something free like awstats (look on sourceforge)

Alternatively, here are the steps you'll need to follow to do what you want by yourself

find where your log files are
look at one days logs by eye and find patterns and delimiters
once you've worked out how your logs are structured you'll need the php to pull it apart

look at the php manual for

explode()

ereg()

ereg_replace()

str_replace()

strpos()

in_array()

 

[click on fuction names above to be taken to the right page]

the two file functions you maybe interested in are
file()
file_get_contents()

both will get the contents of the file

file_get_contents() reads in the entire file as a string

file() returns an array with each new line in a new element, with the new line still attached (ie each value ends \n)

once you have all this you can post it into your data base

cron for every 15 minutes is
*/15 * * * * path/to/php path/to/your/script.php

happy coding!

If you really want to get cute and do it a cool way, configure Apache to log to a pipe

then have the script on the other end of the pipe split the entry and insert it in your database

forget all about the cron job, have it log right to your DB in realtime

Mutilated1 wrote:

If you really want to get cute and do it a cool way, configure Apache to log to a pipe

then have the script on the other end of the pipe split the entry and insert it in your database

forget all about the cron job, have it log right to your DB in realtime

tell me more......

configure Apache to log to a pipe...........?

OK, you know how you specify a log file with the CustomLog directive...

eg:


#in you .htaccess file
CustomLog /some/file/name some_format


???


replace the filename with a pipe

eg

CustomLog "|/some/progam/name/mylogger" some_format



Now Apache gives mylogger the logfile entry on its standard input... 

eg.


#include <stdio.h>
int main( int argc, const char* argv[] )
{

// party on!   for argv[0] is your log file entry

}

Wouldn't have to be C, just has to be a program that accepts the standard input

Of course if you wanted to be really really cute about it, you make your C program wait on a named pipe so Apache is not spawning it in a new process and its not connecting to the database everytime.  That would be safer too, since if Apache doesn't spawn the process, there is no chance of someone upsetting your apple cart and getting access to Apache's UID - your process that listens on the pipe can run as a user with low privlidge.