Skip navigation links

iCrawl 1.0-RC3-SNAPSHOT API

Data model 
Package Description
de.l3s.icrawl.domain
Domain objects.
de.l3s.icrawl.domain.api
Domain classes related to API fetching.
de.l3s.icrawl.domain.crawl
Crawl record entities.
de.l3s.icrawl.domain.evaluation
Evaluation entities
de.l3s.icrawl.domain.event
Events occuring during a crawl.
de.l3s.icrawl.domain.specification
Crawk specification entities.
de.l3s.icrawl.domain.status
Status representation of different tasks.
de.l3s.icrawl.domain.support
Internal support classes for data access.
Data access layer 
Package Description
de.l3s.icrawl.repository
Repository interfaces
de.l3s.icrawl.service
Domain services
Crawler 
Package Description
com.sharethis.textrank  
de.l3s.icrawl.crawlmanager
Crawler control implementation
de.l3s.icrawl.crawlmanager.step
Steps of the crawl execution.
de.l3s.icrawl.online
Analysis of crawled documents
de.l3s.icrawl.online.singlerank
Implementation of the SingleRank algorithm for keyword extraction.
API Crawler 
Package Description
de.l3s.icrawl.api
Interfaces related to API crawling.
de.l3s.icrawl.apifetcher
Retrieve data through Web platform APIs.
User interface 
Package Description
de.l3s.icrawl.ui
Web user interface
de.l3s.icrawl.ui.format  
de.l3s.icrawl.ui.wayback
Wayback Machine integration with the CrawlDB.
de.l3s.icrawl.ui.wizard
Server-side code for the Crawl Specification Wizard
Other Packages 
Package Description
de.l3s.icrawl  
de.l3s.icrawl.adblock
Code to load filter lists for AdBlock Plus and apply them to filter out links to advertisement pages.
de.l3s.icrawl.data.specification  
de.l3s.icrawl.evaluation
Evaluation code
de.l3s.icrawl.evaluation.twitterfetcher
Evaluation baseline that only downloads tweeted URLs.
de.l3s.icrawl.export.warc
Export of crawled documents as WARC files.
de.l3s.icrawl.process
Management of external processes
de.l3s.icrawl.process.impl
Management implementation of external processes
de.l3s.icrawl.util
Helper classes.
org.apache.nutch.util  
Skip navigation links

Copyright © 2014–2015 L3S Research Center. All rights reserved.