笔记 by 车东

AWStats 6.9发布: 补充中文搜索引擎定义和配置样例下载

· 日志分析|AWStats

📌 TLDR

AWStats 6.9 发布后补充中文搜索引擎与蜘蛛定义,并把本站的配置样例打包成 lib.tgz。作者强调 AWStats 用起来不必懂 Perl,改配置即可,与 Google Analytics 搭配就是一套轻量级基础统计。同期向上游提交的补丁包括中文语言包、中文浏览器/蜘蛛、国内搜索引擎定义,以及 clf 日志里以非 / 开头时 robots.txt 解析出错、$LIMITFLUSH 可配置等问题。

Eldy赶在新年前把AWStats 6.9发布了: 主要的蜘蛛定义和搜索引擎定义修改以及本站的配置样例我已经打包在这里;AWStats虽然是perl写的,但是基本上要用起来不需要对perl熟悉,主要是配置的修改,并且可以适用于于大部分网站的流量,结合Google Analytic统计作为网站状况的轻量级基础统计解决方案;

相关的更新也已经提交: 欢迎各位补充,争取在下一个版本中发布 AWStats - Patches - 4 items 1569229 Simplified Chinese language file update 1569201 top Chinese browser and robot update 1569151 TOP Chinese local search engines update 2499455 robots.txt: clfmerged log files maybe not start with / AWStats - Feature Requests- 2 items 2498163 configurable $LIMITFLUSH and increase default value to 50000 706297 IIS timezone:change the timeline instead of change time

完整的diff附后: 包含了awstats.pl本身的2个小修改; 1 针对大量URL:增大$LIMITFLUSH减少临时文件I/O; 2 针对泛域名型应用的修改:使用clfmerge -b合并后的日志无法匹配"GET /robots.txt" (因为被clfmerge拼上域名,变成了 “GET http://foo.example.com/robots.txt");

cvs diff: Diffing .
Index: awstats.pl
===================================================================
RCS file: /cvsroot/awstats/awstats/wwwroot/cgi-bin/awstats.pl,v
retrieving revision 1.926
diff -r1.926 awstats.pl
46c46
<   5000;   # Nb of records in data arrays after how we need to flush data on disk
---
>   50000;   # Nb of records in data arrays after how we need to flush data on disk
10385c10385
<       my $regrobot       = qr/^\/robots\.txt$/i;
---
>       my $regrobot       = qr/\/robots\.txt$/i;
cvs diff: Diffing lang
cvs diff: Diffing lang/tooltips_f
cvs diff: Diffing lang/tooltips_m
cvs diff: Diffing lang/tooltips_w
cvs diff: Diffing lib
Index: lib/robots.pm
===================================================================
RCS file: /cvsroot/awstats/awstats/wwwroot/cgi-bin/lib/robots.pm,v
retrieving revision 1.53
diff -r1.53 robots.pm
356a357
> 'yahoo!\sslurp\schina',
476a478
> 'soso',
677a680,688
> 'blogbus',
> 'QBikSpider',
> 'Zwl.Rss.Spider',
> 'BBReaderFeedSpider',
> 'yeti',
> 'doubanbot',
> 'moreoverbot',
> 'xianguo',
> 'blogvibebot',
998a1010
> 'yahoo!\sslurp\schina','[Yahoo! Slurp China](http://misc.yahoo.com.cn/help.html)',
1115a1128
> 'soso','[Soso Spider](http://help.soso.com/webspider.htm)',
1319a1333,1341
> 'blogbus','[BlogBus](http://www.blogbus.com)',
> 'QBikSpider','QBikSpider',
> 'Zwl.Rss.Spider','Zwl.Rss.Spider',
> 'BBReaderFeedSpider','BBReaderFeedSpider',
> 'yeti','[Yeti](http://help.naver.com/robots/)',
> 'doubanbot','[doubanbot](http://www.douban.com)',
> 'moreoverbot','[Moreoverbot](http://www.moreover.com)',
> 'xianguo','[XianGuo](http://www.xianguo.com/)',
> 'blogvibebot','BlogVibeBot',
Index: lib/search_engines.pm
===================================================================
RCS file: /cvsroot/awstats/awstats/wwwroot/cgi-bin/lib/search_engines.pm,v
retrieving revision 1.46
diff -r1.46 search_engines.pm
288a289
> 'image\.baidu\.com',     # baidu image search
296a298
> '\.youdao\.com',       # powered by NetEase
503a506
> 'image\.baidu\.com','baidu_image',
508a512
> '\.youdao\.com','yodao',
768a773
> 'baidu_image','(wd|word)=',
774c779,780
< 'soso','q=',
---
> 'yodao','q=',
> 'soso','w=',
996a1003
> 'baidu_image','[Baidu Image](http://image.baidu.com/)',
1001a1009
> 'yodao', '[YoDao](http://www.youdao.com/)',

💬 历史评论

以下 2 条评论从旧版 Movable Type 博客迁移而来。 原始评论时间:2002–2016 年。

Yuki · 2009-03-11 12:47

大大你好,在awstats的 參考資訊: 來源網址 … 由那些搜尋引擎轉介 會看到 eg 1)Yahoo 2)Google, 這是否包括從 com.cn or .com or .hk的yahoo,

還是可分出由什麼地區的Yahoo 轉介?

謝謝


你好 · 2009-08-04 18:00

WIN平台下用GNUWIN32打PATCH打不上,怎么办呢? youdao的蜘蛛手工怎么加呢? 谢谢

Index: search_engines.pm
===================================================================
RCS file: /cvsroot/awstats/awstats/wwwroot/cgi-bin/lib/search_engines.pm,v
retrieving revision 1.46
diff -r1.46 search_engines.pm
288a289
> 'image\.baidu\.com',     # baidu image search
296a298
> '\.youdao\.com',       # powered by NetEase
503a506
> 'image\.baidu\.com','baidu_image',
508a512
> '\.youdao\.com','yodao',
768a773
> 'baidu_image','(wd|word)=',
774c779,780
 'yodao','q=',
> 'soso','w=',
996a1003
> 'baidu_image','Baidu Image',
1001a1009
> 'yodao', 'YoDao',