Query Details
| Query Subject: |
Looking for a Web Crawler for Corpus Analysis
|
|
| Author: | Ana Popescu | |
| Submitter Email: | click here to access email | |
| Linguistic LingField(s): |
Computational Linguistics
|
|
| Query: |
Dear All,
I would like to know if there is a web crawler that could download websites in text format - I have a list of aprox. 100 links from which I would like to collect the text (not the pdfs) and then take the .txt files and run them in a concording programme (I already have Wordsmith and another one). I want to be able to get word lists, but also to be able to find the file where a particular word originates - this is why I need the text files. I am interested in a crawler suitable for OS/Windows. Also, I want the crawler to be able to download the sites recursively, if asked to do so. I have found different free software of this kind on the Internet but they don't do everything I need. Thanks. |
|
| LL Issue: | 23.1026 | |
| Date posted: | 29-Feb-2012 | |
|
Back |
||
|
|
||
|
Sums main page
|
||
|
About LINGUIST
|
Contact Us
|
Report Broken Link
While the LINGUIST List makes every effort to ensure the linguistic relevance of sites listed on its pages, it cannot vouch for their contents. |
|
Business Plan,Business Ideas,Advanced Energy,High Technology,Healthy Diets,Healthy Foods,Games Guides,Games Cheats,Travel Guides,Travel Tips,Study Skills,Study Tips,Health Tips,Health Guides,Jewelry Stores,Jewellery UK Online,Digital Camera Reviews,Digital Camera Buying Guide,Replica Handbags,Replica Bags,Jackets on Sale,Jackets Clearance,WoW Gold,Cheap WoW Gold,Buy WoW Gold,WOW Gold,Swtor Credits