Original Bovespa's API is dead

[nihey]

I need to find another way to scrape the data.

[carlopires]

Do we have some way to get data from B3 already?

[nihey]

As of now, there is no open current way to scrape it that I know of.

It seems there are however some paid alternatives, like: Bolsa Financeira

Upon doing some search, there seems to be way, this page gets it using a WebSocket protocol, the data seems to appear like this:

There is also this API, it seems to return just stock codes, but I do not know the full extent of it:

$ curl "https://data-bovespa.tradingview.com/search/?text=ABEV3&exchange=BMFBOVESPA&type=&domain=bovespa" -H "Origin: https://s.tradingview.com"

Whenever I get some free time, I'll try to migrate node-bovespa to use this (it won't be as smooth wrapping a REST API, but it is doable).

[carlopires]

This is weird. TradingView uses data from ICE data services. My main concern is due to the reliability of data, so I wanted to have data as close as possible from bovespa's services.

[lenilsonjr]

Do you guys got any progress on finding another source for the data?

Alpha Vantage seems to be a great alternative. It is free (although it needs registration to obtain a API Key), and have quotes from B3 (with some delay, of course). It is also worth noting that some of the assets listed on B3, like subscription rights, are not available in Alpha Vantage.

And of course, depending on your goal, it might not be a good idea to rely on a third party for stock quotes.

[nihey]

@lenilsonjr Thanks for the information, Alpha Vantage might be the easiest way to got from now. I'll work on it as soon as possible.

[djalmaaraujo]

@lenilsonjr @nihey I could not find B3 quotes in Alpha. How you should pass in the URL? CSNA:B3? This is not working for me

[patrickbard]

@lenilsonjr @nihey I could not find B3 quotes in Alpha. How you should pass in the URL? CSNA:B3? This is not working for me

Don't know if it's still relevant to you, but I could get appending .SA to the stock symbol, similar how Yahoo Finance does.

Example:

https://www.alphavantage.co/query?...&symbol=PETR4.SA)

[nihey]

Hey everyone, I'm planning to make this project alive once more.

I've been thinking about the plans on how to do it and found out that we can download historical series on these files.

My plan to is to make an API out of it for historical data querying. After doing it, the plan is to offer it publicly (or self hosted via the package).

Once the historical data API is done, I can add a realtime API via webscraping from various sources. It may be hard to maintain it, but we'll see the best way to handle it once we reach this point.

[nihey]

I've managed to setup a new API:

https://bovespa.nihey.org/api/quote/ABEV3/2018-02-14

I'll rewrite the code for node-bovespa so that it uses this brand new API, until then, I recommend using this API directly for whoever needs consuming this data.

Note: it currently do not have realtime data, just historic ones.

[djalmaaraujo]

Hey, this is awesome. I just read your email and I am all up to make this project be big. How can I help? Do we have a roadmap?

[nihey]

@djalmaaraujo Great to see you're interested 😄 !

I'll get everything organized this week and possibly create issues, but as a brief explanation on everything I wanted for this project, we should add:

• Extract realtime data via webscraping from some of these platforms

  • https://finance.yahoo.com/quote/abev3.sa?ltr=1
  • https://www.tradingview.com/symbols/BMFBOVESPA-ABEV3/
  • https://www.infomoney.com.br/ambevsa-abev3
  • https://economia.uol.com.br/cotacoes/bolsas/acoes/bvsp-bovespa/abev3-sa/

• Unit testing to receive notifications when the realtime API stop working because one of those sites changed.

• Include the realtime data into the API

• regularly update the historic series with official data from bovespa

• a CLI utility to extract data from the API

# Something like:
$ bovespa ABEV3 -d 2018-01-08
<outputs the data>

• an easy way to access the data

// Something like:
const bovespa = require('bovespa');
bovespa.get("ABEV3", "2018-01-08").then(...)

⬆️ That would be it for now, but I'm open for suggestions too.

[djalmaaraujo]

@nihey A few days ago I received an email for a legal department of XP and a while ago also from bank safra, because I had open source projects with their name or part of it. I am pretty sure this will be a near future for this project as-well, which sucks.

Also, I really love the idea but while I am very interested I am on zero time now, so I am sorry if I passed the wrong impression in my last comment.

I think the roadmap is great but I would ignore the CLI for now since this is not very used compared to an API. That's my opinion.

I will try to find time for this anytime soon. tks

[nihey]

@djalmaaraujo

A few days ago I received an email for a legal department of XP and a while ago also from bank safra, because I had open source projects with their name or part of it. I am pretty sure this will be a near future for this project as-well, which sucks.

That sucks a lot, I really hope that if this happens here, it takes a long time to happen.

Also, I really love the idea but while I am very interested I am on zero time now, so I am sorry if I passed the wrong impression in my last comment.

No problems!

I think the roadmap is great but I would ignore the CLI for now since this is not very used compared to an API. That's my opinion.

I think you're right, although the CLI can be quite handy sometimes, it is not nearly as useful as the API
I should definitely add some documentation not only for the node API, but for the REST API too, and this should be the main focus for the project (ideally aiming into the real time API).

I will try to find time for this anytime soon. tks

Alright, thanks!

[krolow]

I was searching for some solution to get real time and endup here...

Any thoughts in get real-time data?

I'm doing one dashboard for myself my current solution is using google sheet, is not the best one as it does not reflect the current stock price (they say it can be like 20min delay), but at least I get something close to the current stock price like using the formula =GOOGLEFINANCE("ITUB4"; "price")

[krolow]

Maybe for historical data, you can use the same? I think it's going to be pretty easy to maintain...