我希望從以下網站抓取一個包含 4000 多行的表格：
https://www.nasdaq.com/market-activity/stocks/aapl/institutional-holdings
如果可能的話，我最好需要有人來展示如何使用 Nasdaq api。我相信我通常使用 webscrape（使用 beautifulSoup）的方式對于這項任務來說效率非常低。
謝謝！
uj5u.com熱心網友回復：
該表是分頁的，每一頁都是一個新的 XHR 呼叫，帶來 15 條新記錄（由以前的條目抵消）。讓我們利用我們的優勢來操縱 url - 比如說，一次請求 7k 條記錄，偏移量為 0（總共大約 4k 個條目）：
import requests
import pandas as pd
headers = {
'accept': 'application/json, text/plain, */*',
'origin': 'https://www.nasdaq.com',
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Safari/537.36'
}
pd.set_option('display.max_columns', None)
pd.set_option('display.max_colwidth', None)
url = 'https://api.nasdaq.com/api/company/AAPL/institutional-holdings?limit=7000&offset=0&type=TOTAL&sortColumn=marketValue&sortOrder=DESC'
r = requests.get(url, headers=headers)
df = pd.json_normalize(r.json()['data']['holdingsTransactions']['table']['rows'])
print(df)
結果：
ownerName date sharesHeld sharesChange sharesChangePCT marketValue url
0 VANGUARD GROUP INC 06/30/2022 1,277,319,054 7,323,304 0.577% $191,214,662 /market-activity/institutional-portfolio/vanguard-group-inc-61322
1 BLACKROCK INC. 06/30/2022 1,028,688,317 1,055,430 0.103% $153,994,641 /market-activity/institutional-portfolio/blackrock-inc-711679
2 BERKSHIRE HATHAWAY INC 06/30/2022 894,802,319 3,878,909 0.435% $133,951,907 /market-activity/institutional-portfolio/berkshire-hathaway-inc-54239
3 STATE STREET CORP 06/30/2022 598,178,524 -15,673,750 -2.553% $89,547,325 /market-activity/institutional-portfolio/state-street-corp-6697
4 FMR LLC 09/30/2022 350,900,116 6,582,142 1.912% $52,529,747 /market-activity/institutional-portfolio/fmr-llc-12407
... ... ... ... ... ... ... ...
4397 VERSOR INVESTMENTS LP 09/30/2022 0 -5,171 Sold Out /market-activity/institutional-portfolio/versor-investments-lp-1015149
4398 WALLEYE CAPITAL LLC 06/30/2022 0 -44,561 Sold Out /market-activity/institutional-portfolio/walleye-capital-llc-1069483
4399 WALLEYE TRADING LLC 06/30/2022 0 -65,383 Sold Out /market-activity/institutional-portfolio/walleye-trading-llc-733607
4400 WARATAH CAPITAL ADVISORS LTD. 09/30/2022 0 -31,149 Sold Out /market-activity/institutional-portfolio/waratah-capital-advisors-ltd-901912
4401 WINSLOW CAPITAL MANAGEMENT, LLC 06/30/2022 0 -2,386 Sold Out /market-activity/institutional-portfolio/winslow-capital-management-llc-64122
4402 rows × 7 columns
轉載請註明出處，本文鏈接：https://www.uj5u.com/caozuo/532676.html