gerenuk

Hi there,

We just received an email from Mercury that they are closing down operations from August 22, 2024 because we are resident of the country which is blacklisted.

We've been using Mercury from last 4 years and processed in millions for transactions.

What other banking platforms do you recommend?

Thanks.

Hi,

While building the application feature, how do you plan to work on the specific feature? Like creating the user-stories, gathering requirements?

I am looking to improve my thought-process and cover most of the edge-cases edge-cases as a solo developer.

Hi,

We're currently trying to build a proper testing process.

Just having some unit tests and running the app a couple of times before shipping to production doesn't cut it anymore :)

How do you organize the testing process in your company?

What tools and procedures do you use?

Thanks

Hi,

We have a dataset around 150 million URLs and other meta data in ElasticSearch and looking for an efficient way to identify the duplicate URLs/titles from our dataset. Used ElasticSearch term aggregation but it becomes very slow and returns only 10,000 URLs and most of the time it misses the URLs.

Currently, we have a redis with Sorted Sets, before any indexing URL, we look for the into redis set.

Options we have explored:

1. Clickhouse, storing all the URL and running aggregation etc. on it later on? 2. Storing the URLs in redis along with bloomfilter.

If you have worked on a similar thing, would love to hear your feedback.

Thanks.

Hey everyone,

Looking for suggestion and recommendations, what would be your ideal choice if you are about to build a backlinks database and running later on PageRank algorithm on it. Our use case is to improve the search relevancy depending on the URL rank.

Considered Storage DBs:

1. Clickhouse 2. ElasticSearch (team is experienced with it, but finding a duplicate URLs etc. over a huge dataset is still clumsy in ElasticSearch) So thought of going with the Clickhouse.

Thanks.

Hi everyone, We are currently using text classification model trained on a labelled data set consisting of almost 500 hierarchical categories , For example a news article is labelled as News/Business News , now what we want to do is extract sub-topics from this main category such as Stock Market , Trading News, Business Investment, Angel Investments etc . So what approach would be recommended to handle this kind of problem ?

Thanks

Hi everyone,

We are currently using text classification model trained on a labelled data set consisting of almost 500 hierarchical categories , For example a news article is labelled as News/Business News , now what we want to do is extract sub-topics from this main category such as Stock Market , Trading News, Business Investment, Angel Investments etc . So what approach would be recommended to handle this kind of problem ?

Thanks

A common scenario is workers/crawlers are running on more than 100+ machines with supervisord with different configuration? What would you suggest for monitoring and knowing what kind of workers are running on each machine?

Thanks.

Hey everyone!

We are trying to solve a problem where we need to classify the articles into the right categories.

Currently, using a FastText to train a model with 100,000 articles categorized into 600 categories. The loss seems to be converging but the precision is not going up, another thing that requires clarification is that can we use pre-trained Wikipedia English embeddings to categorize text.

What would you recommend using FastText or some other algorithm/approach towards this problem?

Any suggestion/ideas would be appreciated.

Thanks.

Hello Everyone!

Currently, using ElasticSearch for storing the meta data and other raw data information but it is a very small scale around 500,000 domains.

I have been tasked to scale it to 20-40 million domains and storing their internal/external links while building a page rank/domain authority score for each domain which we are adding to our database.

What do you guys suggest/recommend for storing this data at a very large scale as web page internal links/external links will be stored which will lead it over 100M-1B links database?

Any kind of feedback/suggestion would be appreciated.

Thanks.