网站数据抓取-创新互联

创新互联网站建设提供从项目策划、软件开发,软件安全维护、网站优化(SEO)、网站分析、效果评估等整套的建站服务,主营业务为成都做网站、网站制作,成都App定制开发以传统方式定制建设网站,并提供域名空间备案等一条龙服务,秉承以专业、用心的态度为用户提供真诚的服务。创新互联深信只要达到每一位用户的要求,就会得到认可,从而选择与我们长期合作。这样,我们也可以走得更远!>1:
private bool ImportSubjectQuesData(int bankId, int categoryId, int curpage, int pagesize) { string postData = "bankid=" + System.Web.HttpUtility.UrlEncode(bankId.ToString(), System.Text.Encoding.ASCII); postData+= "&categoryid=" + System.Web.HttpUtility.UrlEncode(categoryId.ToString(), System.Text.Encoding.ASCII); postData+= "&curpage=" + System.Web.HttpUtility.UrlEncode(curpage.ToString(), System.Text.Encoding.ASCII); ....参数省.... string url = "http://www.*****.com/Web/Handler1.ashx?action=queslistquery"; byte[] byteArray = Encoding.UTF8.GetBytes(postData); var webRequest = (HttpWebRequest)WebRequest.Create(new Uri(url)); webRequest.Method= "post"; webRequest.ContentType= "application/x-www-form-urlencoded"; webRequest.ContentLength= byteArray.Length; Stream newStream= webRequest.GetRequestStream(); newStream.Write(byteArray,0, byteArray.Length); newStream.Close(); //接收HttpWebResponse response = (HttpWebResponse)webRequest.GetResponse(); StreamReader php= new StreamReader(response.GetResponseStream(), Encoding.GetEncoding("utf-8")); string result = php.ReadToEnd(); int Index = result.IndexOf("###"); var retCount = DeserializeObject(result.Substring(0, Index)); int Count = Convert.ToInt32(retCount.quescount.Value); int intPagesize = Convert.ToInt32(pagesize);数int pageCount = Count % intPagesize == 0 ? Count / intPagesize : Count / intPagesize + 1; result= result.Substring(Index + "###".Length); var questlt = DeserializeObject(result); //数据录入return InserData(questlt); }

2:
string url = txtshiUrl.Text + i;
var webRequest = (HttpWebRequest)WebRequest.Create(new Uri(url));
webRequest.Method = "GET";
webRequest.UserAgent = "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)";
webRequest.ContentType = "text/html; charset=gbk";
webRequest.KeepAlive = true;
webRequest.Referer = url;
webRequest.CookieContainer = new CookieContainer();
HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();
responseStream = webResponse.GetResponseStream();
StreamReader streamReader = new StreamReader(responseStream, Encoding.GetEncoding("gbk"));
string shtml = streamReader.ReadToEnd();网站数据抓取

本文标题:网站数据抓取-创新互联
地址分享:http://myzitong.com/article/dpsiep.html