The library supports multiple ways of loading robots.txt files

Loading from a Uri
var robots = new Robots.Robots();
robots.Load(new Uri("http://www.bing.com/"));


Loading from a stream
Stream stream = ...
var robots = new Robots.Robots();

robots.Load(stream, new Uri("http://www.bing.com/"));

robots.Load(stream, "http://www.bing.com/");


Loading from a string
string content = ...
var robots = new Robots.Robots();

robots.LoadContent(content, new Uri("http://www.bing.com/"));

robots.LoadContent(content, "http://www.bing.com/");


Loading from a TextReader
TextReader reader = ...
var robots = new Robots.Robots();

robots.Load(reader, new Uri("http://www.bing.com/"));

robots.Load(reader, "http://www.bing.com/");

Last edited Sep 14, 2010 at 1:10 PM by saguiitay, version 1

Comments

No comments yet.