Links

Content Skeleton

This Page

Previous topic

SCRAPE

Next topic

Reviewing scrape layout in XMLDB and file system

SCRAPE BUILD.XML

Typically used via scrape- functions, however for debugging usage at ant level is still possible:

ant info
ant -Dpos=00101         -Dopt=noindv
ant -Dpos=00300         -Dopt=noindv
ant -Dpos=comb,base,all -Dopt=noindv
ant -Dpos=comb          -Dopt=noindv
ant -Dhfag.scrape.date=20060122-nohtm make-tgz

FUNCTIONS

info
dump parameters
get-sidebar
the sidebar provides the top level links (no longer used ? now donw by scrape-sidebar)
ls-sidebar
look at the sidebar, via xmllint (as vi chokes on it)
ls-log
formatted listing of the scrape log
find
look for .log , this shows the progress of the scrape by looking at the timestamps on the logs from each of the top level links
scrape
The default target, runs scrape.pl traversing the links in the sidebar doing recursive scrapes from each, depending on the base.apt
checklog
list GETs with error status in the log by running checklog.pl on the log
dev
runs dev.pl
rescrape
runs rescrape.pl
tlog
transform the log into something more presentable
look
display the docbook log thru cocoon url (NOT WORKING?)
crucial
display environment and timestamps on crucial files
openlog
open the local scrape logfile
present
transform the scrape log into docbook (tlog followed by look)