Jip J. Dekker
|
75c0be1fea
|
Added tests for the pipline
|
2014-06-04 16:50:14 +02:00 |
|
Jip J. Dekker
|
c48c4ec697
|
None pipeline doesn't need a set
|
2014-06-04 16:09:55 +02:00 |
|
Jip J. Dekker
|
d4a0ffdff3
|
Optimized imports
|
2014-06-04 12:01:05 +02:00 |
|
Jip J. Dekker
|
ecee4a5f45
|
Merge branch 'develop' of github.com:Recondor/Fourmi into develop
|
2014-06-01 20:30:50 +02:00 |
|
Jip J. Dekker
|
aac0a7c79c
|
References to the main Scrapy documentation
|
2014-06-01 20:29:51 +02:00 |
|
Jip J. Dekker
|
f81b1c9500
|
Fixed a typo
|
2014-06-01 20:25:46 +02:00 |
|
Jip J. Dekker
|
f7d0fb4a45
|
Added documentation to the basic Source
|
2014-06-01 20:24:54 +02:00 |
|
Jip J. Dekker
|
c27a875d68
|
Parser/Source consistency
|
2014-06-01 20:18:03 +02:00 |
|
Jip J. Dekker
|
3499946e97
|
Fixed a typo
|
2014-06-01 20:15:15 +02:00 |
|
Jip J. Dekker
|
c4876f029b
|
Added documentation to the FourmiSpider
|
2014-06-01 20:14:47 +02:00 |
|
Jip J. Dekker
|
ace4393a8f
|
Merge branch 'feature/NIST-source' into develop
|
2014-05-23 13:01:06 +02:00 |
|
Jip J. Dekker
|
0e7e4cbe61
|
Merge branch 'develop' of github.com:Recondor/Fourmi into develop
|
2014-05-22 12:17:56 +02:00 |
|
Jip J. Dekker
|
98f91a1aa9
|
Added a pipeline to replace None values with empty strings
|
2014-05-22 12:15:43 +02:00 |
|
Rob tB
|
6ce5ff2335
|
replaced name variable with summary variable
|
2014-05-21 10:40:44 +02:00 |
|
Rob tB
|
6cd8edaf22
|
included summary variable in call to transition_table, antoine table and generic table
|
2014-05-21 10:36:42 +02:00 |
|
Rob tB
|
c0af24644b
|
added summary variable in parse()
|
2014-05-21 10:31:19 +02:00 |
|
Rob tB
|
429ffd7422
|
renamed tables to table in parse()
|
2014-05-21 10:28:54 +02:00 |
|
Rob tB
|
95565042ca
|
removed unused variable symbol_table from parse_transition_table
|
2014-05-21 10:22:03 +02:00 |
|
RTB
|
81719a38fb
|
Added comments for the class and functions
|
2014-05-20 19:32:06 +02:00 |
|
RTB
|
472aae86be
|
synonyms are now scraped
|
2014-05-17 19:32:20 +02:00 |
|
RTB
|
b46c7a309d
|
if synonym name matched in search instead of primary name, emit primary name as synonym
|
2014-05-17 14:21:11 +02:00 |
|
RTB
|
afc1106838
|
NIST now logs an error if chemical name is not found
|
2014-05-17 14:11:03 +02:00 |
|
RTB
|
56ee6b1ad3
|
added ignore list
|
2014-05-17 14:09:10 +02:00 |
|
Rob tB
|
98f58ea4e2
|
added scraping for generic info except for synonyms
|
2014-05-15 14:29:28 +02:00 |
|
Rob tB
|
50c79e3b1f
|
conditions in name (split by ' at ') are now moved to condition field for individual value page and aggregate data table
|
2014-05-14 13:44:43 +02:00 |
|
Jip J. Dekker
|
c7ad35239e
|
Merge pull request #4 from Recondor/feature/wikipediafixes
Feature/wikipediafixes
|
2014-05-13 21:37:53 +02:00 |
|
Bas Vb
|
b54568bab0
|
Small fixes
|
2014-05-13 16:18:32 +02:00 |
|
Jip J. Dekker
|
afaa0d903f
|
Merge pull request #3 from Recondor/feature/chemspider-parser-fixes
Feature/chemspider-parser-fixes
|
2014-05-09 23:06:43 +02:00 |
|
RTB
|
7e984b60d8
|
added uncertainty to results from scraping individual data points urls
|
2014-05-09 14:24:08 +02:00 |
|
RTB
|
775a920b9b
|
NIST scraper now handles urls with individual data points
|
2014-05-09 13:00:22 +02:00 |
|
RTB
|
5e067fd572
|
altered scraping of aggregate data to test for and request url to individual data points
|
2014-05-09 12:36:54 +02:00 |
|
Jip J. Dekker
|
f193aac24a
|
Fixed Duplicate Pipeline + rename
|
2014-05-08 15:45:42 +02:00 |
|
Rob tB
|
74dddace88
|
removed logging of Result objects in debug messages because pointless
|
2014-05-08 15:42:53 +02:00 |
|
Jip J. Dekker
|
2e654255c5
|
Added documentation to the pipeline.
|
2014-05-08 15:35:18 +02:00 |
|
Rob tB
|
f6fa5e8adf
|
fixed scraping of unit for two kinds of tables
|
2014-05-08 15:22:48 +02:00 |
|
Jip J. Dekker
|
2fcec009bb
|
Added an Pipeline to deal with attribute selection
|
2014-05-08 15:20:48 +02:00 |
|
Rob tB
|
151f1988a1
|
added function to scrape table for Antoine equation parameters
|
2014-05-08 14:54:30 +02:00 |
|
RTB
|
7abb491d3f
|
added function for most generic tables
|
2014-05-07 22:08:43 +02:00 |
|
RTB
|
10dd74e026
|
added function to scrape transition tables
|
2014-05-07 21:58:52 +02:00 |
|
RTB
|
85595ecf35
|
created function to start scraping the aggregate data table
|
2014-05-07 18:12:08 +02:00 |
|
RTB
|
95e24f9c44
|
added code to recognize various table formats
|
2014-05-07 17:09:42 +02:00 |
|
RTB
|
9c80f291b6
|
search NIST exclusively for phase change data
|
2014-05-07 13:27:22 +02:00 |
|
Rob tB
|
930eb6cad5
|
NIST now scrapes the symbol table for later use
|
2014-05-04 21:20:46 +02:00 |
|
Rob tB
|
0cec4bd2d8
|
new_compound_request now returns a Request with a searh URL
|
2014-05-04 21:06:39 +02:00 |
|
Bas Vb
|
03e652d454
|
Wikipediaparser now works on chemboxes as well
|
2014-05-01 16:05:37 +02:00 |
|
Rob tB
|
e1e507f745
|
added several required imports
|
2014-05-01 15:30:28 +02:00 |
|
Bas Vb
|
f8d390d3e6
|
Starting with fixing the wikiparser
|
2014-05-01 15:04:11 +02:00 |
|
Rob tB
|
95e9675605
|
created stub for NIST parser
|
2014-05-01 14:57:09 +02:00 |
|
RTB
|
ca0a22ae7b
|
Added test for empty values on properties in ChemSpider ExtendedCompoundInfo API
|
2014-05-01 12:14:52 +02:00 |
|
RTB
|
73753a6294
|
chemspider source now handles vague search requests
|
2014-05-01 12:04:44 +02:00 |
|