Apache - Access Log parsing in php?

lech

Long story short, no cgi-bin, so I'm looking for a decent stats package which can parse through several megs of apache access and maybe even error logs for one if not all of my domains in the form of a nice php package. I've been searching through sourceforge, google and nearly everywhere in between for the (keyword, pay attention) free open-source php based parser/reporter. Needless to say what I found was either a whole lot of nothing, a bunch of unconvincing alpha-ware or just general code classes to build my own, and I am tempted to at this point if it comes down to it. So besides the general mint/shortstats/slimstats type of packages, what out there do any of you know coded in php can do apache access logs? Suggestions or am to be convinced a project like this doesn't exist in php? Speed isn't a concern, parsing is. :) Discuss.

Minisweeper

I use awstats on my server but i forget whether its php or something else. I'm pretty sure you can get it running with command line access though (i.e. no cgibin)

*puts his thinking cap on.

jakob_r

maybe logfiler (PHP flatfile server access logger) might do what you're looking for. The page is in German but if you need a hand with the explanation, mail me. It logs server access info to a flatfile and presents you with Referrer, hits and infos to User, Referrer/Domains, URL, Browser&OS, Search terms/links.

lech

Mini, I've looked, awstats is good and fast, however it's perl/cgi-bin'ned, it along with similar are sadly all cgi-bin/perl based :\ jakob, while this looks nice, it really reminds me of a better referrer tracker I coded a while ago which spit out results into CLF flatfile. I'm guessing that this too is something which I'll need to include into all of my pages/scripts, which is not exactly what I want to do. I'm not really interested in referrers primarily, I just want to parse the existing logs :) Nice find though.

Dinoboff

Hello, What is the problem with awstats being a perl script? Is it a security issue? Like said Minisweeper said you can use awstats on the command line and access the stats in an html output.

lech

While I have shell access, it's a few extra unecessary hoops to jump through, but if I must, I suppose that's the only way. I asked for my cgi-bin to be disabled, so that's out of the question as I've had a nightmare running within it before and don't wish to encounter that again. So just looking for a nice php alternative if possible.

Dinoboff

while most of the tutorial about awstats talk about the cgi-bin use of it, it is quite easy to use awstats to create static html pages.

The best tutorial I have read have been the one from dotvoid.com.

Basically, to adapt this tutorial and create static html pages of your stats you will use awstats_buildstaticpages.pl instead of awstats.pl like this:

awstats_buildstaticpages.pl -update -config="www.mysite.com" -awstatsprog="/path/to/awstats.pl" -dir="/var/www/awstats"

and take off in httpd.conf:
"ScriptAlias /awstats "/var/www/awstats/"

and add in the directive for /var/www/awstats/,
"DirectoryIndex awstats.www.mysite.com.html index.html"

Minisweeper

Do you have cron lech? I managed to setup my bros server to run awstats with static pages (couldnt for the life of me work out how to make it do it dynamically at the time) within a reasonable timescale given i had absolutely no idea what i was doing. Then i just cronned it up to run at 3am every day.
I suspect for someone slightly more experienced it wouldnt take a huge amount of time.

A nice php stats thing would be nice but I've never had the need to go looking for one to be honest.

Minisweeper

How come that em tag isnt closing?!

Dinoboff

my mistake lol

Is it a known bug?
Could you have close it for me?

Minisweeper

Yeah, i'm just confused why i couldnt close it for you. *shrugs

Minisweeper

Where the hell did those strong tags come from and why isnt it closed properly?! This is funky.

Dinoboff

I just add, to see... funny?

lech

Yeah, that's one of the things, I kind of like "live" stats getting recompiled and parsed on the fly when I ask for them. And trust me, I've looked over and used AWStats and it's competitors in the past on other servers before. While there's some features about it I like, there's others which are "meh", and it seems a little too bulky given what it's parsing. While you can customize what it parses is pretty standard I think I might try something else until I can actually find something in a language I'm more comfortable with. I've so far managed to collect nearly 20mb of logs (access/error) in common logging format starting since mid-april, and in my spare time I've been doing odd searches for example tid-bits of CLF parsing classes to get myself started. From there, who knows, but maybe with a little help something useful may come of it.

Minisweeper

I'd love to offer to help cause i could do with a side project over summer but I have exams till the 22nd then I'm away first week in july. Oh and I'm really not much use with php as far as OO goes. Then again i should probably learn it.

Maybe i'll find an easy extension to write for vanilla again and try and get my head round it.

I also want to learn some basic c++ so i can write a control app for my projector blind and other stuff which should be near impossible (for me).