kevinfat

In the information retrieval space I am aware of software such as the following: lucene, solr, elasticsearch, sphinx, manticore, etc. As far as I am aware these are based on inverted indices with a bunch of stuff like custom stemmers for the particular language being used. Now this is rather annoying because to make improvements requires handcrafting various customizations for each language.

Is there a way to apply ML to such a problem? I am envisioning the following

* the smallish dataset to be queried on

* a much bigger corpus for teaching the system about the language so we don't need to handcraft customizations for the language such as a stemmer

* queries and labeled best results from the dataset to learn a ranking from

Is any of that possible with neural networks and if so where would one start to learn about what works? Ideally we want something where we can train on ever bigger data without needing to think about stuff like handcrafting feature vectors.

How do startups get their sales transactions into their accounting software (Quickbooks, cloud based software, or whatever is used)?

I assume with your typical web app credit card processing is done with Stripe, Braintree, etc., how should each individual sales transaction make it into the accounting software.

Do they have each sale end up being recorded along in a PostgreSQL database? But then this needs to be exported to the accounting software that is used. It would make a lot of sense for the entire pipeline to be automated obviously.

Really what I'm asking is what is the common practice software architecture to handle a sales end to end?

What current attempts are there to replace email with something secure rather than encryption slapped on after the fact like with email? Desirable properties would be like, for example, Google Wave had a federated model which thus allows you to run your own server which gets around the problem of the feds backdooring web based encrypted solutions like Lavabit.

What infrastructure do news aggregators typically use to do the crawling, parsing, indexing, extracting, storing/format in database, etc.? Are there open source news aggregators so that I can learn how those problems are solved?

If I wanted to make a news aggregator site how do I get a curated list of news sites such as http://www.latimes.com/ http://www.washingtonpost.com/

Its not realistic for me to manually compile a list of them all by myself. How did the popular news aggregator sites build a comprehensive list?

Heroku does not support WebSockets. Which Rails hosting platforms support WebSockets. (Also, if any, which of them give a free mini instance like Heroku does?)