Added deduplication handling and restructured code. Seemingly somewhat unstable. Recommend using slowscrape until further notice
