Solution for Get request for txt file doesn’t work on local host , but works on google colab
is Given Below:
I’m trying to download the txt/csv file from the exchange.
I have identified this url to contain the info.
The code works on google colab but doesnt work on local Jupiter notebook.
The cell just keeps running without results if not interrupted and returns interruption.
The site limits the number of tries.
import requests import pandas as pd import io url="http://www.cffex.com.cn/quote_IO.txt" data = requests.get(url) CSI300_option = pd.read_csv(io.StringIO(data.content.decode('utf-8'))) CSI300_option
The following is the HTTP request I have from the dev tools.
GET /quote_IO.txt HTTP/1.1 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Upgrade-Insecure-Requests: 1 Host: www.cffex.com.cn User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15 Accept-Language: en-GB,en;q=0.9 Accept-Encoding: gzip, deflate Connection: keep-alive
It seems that you need to be able to specify a timeout on the HTTP GET.
This is the minimal code that I use to achieve success:-
import pandas as pd import requests import io URL = 'http://www.cffex.com.cn/quote_IO.txt' with requests.Session() as session: response = session.get(URL, timeout=5) response.raise_for_status() df = pd.read_csv(io.StringIO(response.text)) print(df)