The sad state of NLP tech in the marketplace today

Update: Sinequa has caught up to us in going straight to the answer, EXCEPT we are FASTER and MORE ACCURATE. Don't believe it? Try the first 3 covid queries on our covid virus search at noonean.com and then try them against sinequa's covid search. We blow them away. One good thing about so many companies doing covid search, we finally get the put up or shut up comparisons.

-------------------------------------

Very few products exist which use advanced NLP techniques. We've scoured the marketplace and find product after product lacking.

Here is a brief list of things they do:
 
     Phrase recognition
     Simple named entity recognition
     Part of Speech
     Lemmatization
     User Intent Analysis
     Disambiguation

While it's true that all of these are parts of NLP none of them are ADVANCED NLP techniques. Do they help improve search, yes. But not as much as using advanced techniques.

Here is how one competitor describes their tech:

• Semantic search: uses a variety of signals to understand the user’s intent and handle ambiguity. For example, semantic search understands that when a user searches for “profit,” she would also want to find data sets that reference “net income.” Similarly, if a user searches for “NJ,” they would also return entries for “New Jersey.”
• Lemmatization uses variations on words such as plurals, tenses, genders, hyphenated forms, and more. For example, a search for “running” would return matches for “runs” or “ran.”
• Advanced syntax increases precision through techniques such as phrase search, fielded search, Boolean matching, and proximity search.
• Fuzzy matching increases recall and allows for looser matching. Substring and approximate matches allow users to find data sources when they only have partial information or even incorrect information.

None of this is Advanced NLP. And sadly it's commonplace that companies are pitching that they have NLP tech in their search but it's all very rudimentary. While you can't argue that lemmatization and simple named entity extraction is NOT NLP, because it is, it's really just the first few steps of a processing chain with the higher order functions lopped off and ignored. And why is that? Because they don't really have backgrounds in doing NLP research. 

But it's a successful strategy that gets investment and sales, but really it's a shoddy product. So just like the AI buzz that gets everyone funded, it's quite troubling that such minimal low level tech is representative of what's state of the art. It isn't. 

Shockingly, things are no better at Google, Microsoft, or Amazon regarding their NLP search offerings.  Focused more on User Intent recognition / Sentiment analysis and disambiguation, these products are focused on Chat Bots not Enterprise Search. 

Sinequa just recently demo-ed their tech at KMWorld. First you get a list of documents and then you have to click on the documents to see the analysis of each document.  Noonean is more advanced, we show you the answers DIRECTLY not documents. 

Noonean is designed from the beginning for enterprise scale and works with a parallel index so you can trial the technology without any risk of damaging your current enterprise. The index can reside as a SOLR core on the same machine or a completely different machine. 

Noonean.com brings more advanced NLP techniques to the market.

Companies see the value of automating customer service and think their traditional search technologies are sufficient for the enterprise. But moving to Cognitive Search and Insight engines with Advanced NLP and AI learning will take things to another level and have a profound impact on their bottom lines.  In reality, enterprise search is a first line crude form of knowledge management, and fact bases based on NLP Ontologies are the next level after that. 

User Intent Analysis and NLP - Potato Chips or Computer Chips

One area that is not the sine qua non of NLP but nonetheless a core technique is user intent analysis. This is more critical for areas like building a chat bot or an Alexa, and has some but lesser utility in enterprise search.

So consider the user entered query:
    what is the most expensive chip

Do they mean computer chip or potato chip? How can the machine know? Technically this is a query disambiguation class problem.

There are two types of user intent. One is categorical and the other is free intent.

For categorical intent, a NLP corpus is split into different areas. This is quite common. One person may work in the food side of the org and another in the silicon chip side. Typically there might be 10-30 categorical intents.

To solve for categorical intent, each corpus would get processed in the different areas and statistical maps or 3D spatial similarity maps would be generated. Typically these are generated on NLP generated tokens not the words themselves.  In our case, expensive would have a stronger correlation score to computer chips than potato chips. So that would trigger the user intent of "computers" and query transforms to boost to that would be applied.

The other user intent is more subtle. Which is to boost a query based on historical analysis of the user. So if the user's other query is "what is the fastest ram" then we might deduce that he is seeking computer chips. If the user's other query from history is "what is the saltiest snack" then we might deduce potato chips.

Ah but do we really know the user's intent. What if they've just started a keto diet and all potato chips are verbotten.  So if we start re-writing queries based on analyzed intent that's going to piss a lot of users off!  So for historical intent it's more an art than a science, and the goal is often to extend and nudge the query rather than obliterate it. How much is enough and how much is too much? Generally the intent provided term should be enough to show up in result sets but not dominate. Think of it as a leather clad dominatrix who uses a pillow rather than a whip. err. ok don't think of that. The point is, it's subtle and not overbearing, which is the mistake that's made most often.  Same goes for a chat bot but much worse - Alexa, turn off the lights becomes burn down the house. "sure can do!" replies alexa and turns on the oven.  Subtle. These are guesses after all!   They take a lot of tuning and regression testing to get the right sensibility.

Products which deliver intent analysis without enterprise search integration are clearly targeting chat bots. And while chat bots have their niche use, providing User Intent is also a powerful tool for enterprise NLP search.

bFloat16 for Neural Networks? You win some and you lose some

So you're probably wondering what's the hype with Nervana's processor and the push to adopt bFloat16 format for AI.

One of the main issues is that the extreme performance levels needed for large scale neural networks requires performance in the 50-200 TFLOP range.  Large scale being something like 8-20 billion neuron units.  So there are a few cards that can do such things like Nvidia's new Titan Rcx(?) something or other. But there's a catch. It's only at 16bit. go to full 32 bit precision and performance drops off.

Neural networks rarely need such large numbers. What they need more is precision. Yet bFloat16 only has 7 bits for the fraction, rather than standard 16 bit float which has 10. since each is a DOUBLING of range, those 2 bits mean a big difference in precision. What gives?

Remember that most neural networks store values between 0 and 1. The need for large numbers doesn't exist.

It has to do with the speed of moving huge pipelines of 32 bit precision data in traditional memory into and out of these systems. By removing 2 precision bits it allows conversion from 32 bit float to be much simpler as the exponent is identical. You basically just whack 16 bits off the end.

32 bit floating point

bFloat16

What is the trade off? Well, its the loss of precision in the small numbers. So if you are designing a re-entrant highly convolutional dimensional neural network precision might be much more important to you vs. a vision system which needs huge data pipes into the system.  Since neural network systems use very specialized memory (gddr6 or HMB) which is more expensive, its tough to get 16 or even 24 gigabytes of ram to support the processing. That limits the number of nodes you can express.

bFloat16 is the google pushed format, but is it right for everyone? No it's not. The loss of precision will probably hinder the kinds of problems you can solve with neural networks.  Personally, I'd rather go with precision.

The Importance of Broad Querying and Narrow Indexing

One of the issues we came across with developing our enterprise NLP search engine was the issue of precision. Rather than rank, if the precision level is not met documents simply fall out of query match.

Let's take the case of an object being a predicate object in the query vs being a noun subject in the sentence being indexed.  There would be no match.  So how do we handle this? Well, we can create a broad query - containing the object as both noun subject and predicate object. Now it will match sentences that have either case.

Wait didn't we lose precision? Yep, we sure did. So how does that pass the muster? Well it has to do with two things. The first is many other things are contributing to the match. And all those elements will result in partial scores that add up to a total. The end user experience of the results found will match their mental expectation - they think that queries will resolve symmetry of grammar structure, or as we call them inversions.  There can be models where prepositional clauses dangle from different parts of the sentence. Should that be a match? For enterprise search a large part of this process is tuning and using a "gist" a "gestalt" that things feel ok to the brain. There is no hard and set rule.

Another way to approach broadening is to handle cases where you didn't find the expected document and you realize it was due to a narrow definition.  While you can't simply broaden the case, you can run a test broadening it and see how it effects your regression test set of queries. And then do a bit of spot checking. 

Finally the last issue is that of the near miss. This often happens with the verb. Microsoft accquires Documentum.  Microsoft purchased Documentum.  This might result in a total miss. So how to handle this? The best technique is to run your docset through a processor which determines similarity and clustering. Then you can extend your query with additional terms if there are any within a specified distance.  Again, its a technique that can assist or blow up your query. It takes tuning and time to review many queries.

So if your latest search technology isn't producing the results you expected, remember that broadening the search vs the index is one technique to bring more potential matches into your query rankings.

The Big Bang is a MIS-Interpretation of a Vibrating Universe

Doppler light shift. This one phenom is the basis for the expanding universe. So we assume galaxies are flying apart.

That brought Idiot Hawkings the notion of expansion in reverse - the Big Bang.

But what if doppler light shift is not a result of velocity, but rather, the cumulated effect of vibration pattern waves on light motion.  A sort of standing still movement that accumulates.

In this paradigm, the universe is stable not expanding, and the Big Bang never happened.

Somewhere in my immense vast spare time I have a book in the works "After Einstein, the Vibrationary Model of the Universe" but alas, I'm so stuffed at the moment and there's one or two books ahead of it! So until then you'll have to settle for the interesting but inchoate and lesser treatise on quantum lattice structures (which I now refer to as aethertons) and the earlier writings by Lorentz before he caved to the Einsteinians under pressure. Poor Lorentz.

Why most Suns form as binary systems and why we live in a binary system

Our sun's binary companion - Nibiru - which is in long elliptical orbit many times more vast than Pluto's, swings in every 26,000 years. When it does, an interference pattern forms which is the Shamash, which is shown in many many Akkadian and Summerian carvings. The pyramids and the large granite caskets were shelters from the intense heat and radiation, and flooding. The royals would climb in and spend a month inside with food, and finally their minders, in their last act before dying, would pry the lids and release them. They would enter an Earth unlike anything they could imagine, a vast wasteland, a slate wiped clean of civilization. What little survived would be wild, untamed starting over as apes in the trees. Humans free of knowledge. The few human survivors would build primitive huts, but the world of math and science and technology would be utterly lost.

That is the premise for my new book - The Re-Seeders, which discusses the legends of the Fish scaled bearded ones who re-established culture and science across the globe, whose powers astounded the primitive peoples.

But for now, let's discuss some fundamental physics of star formation. The hydrogen, that's the pure stuff. and that's the driver of the furnace. But what about the impure elements that exist at star formation. For years we assumed they simply remained within the massive star and the gravitational forces formed higher and ever higher order elements with heavier molecular weights.

But what if star formation was more like a crucible, a puddler's boil of lead, where to make a good casting they have to scrape off the slag, the top sludge of impurities.  What if to be a fully functioning furnace of fusion (gee thats amazing alliteration huh?) the star has to eject that impure set of heavier elements. Would they rise to the top or sink to the bottom? A forming star is sparse. So there would be time for the heavier elements to coalesce and slowly gather like a large planet, and then the outward push of heat as the star activates would thrust the heavy particles out perhaps in a massive ball ejection. and at that very moment, achieving purity, the flames of fusion would ignite.

And the cast out ball would get thrown into a wild distant elliptical orbit.  The weight of ten Jupiters in a mass smaller than Jupiter. A kind of metal star. But without fire. Without a heart of fusion. A dark object. That would be very hard to find with a telescope of any kind.

In the past few years astronomers have painstakingly measured the rate at which stars and gas clouds in the outer parts of spiral galaxies are orbiting the center of mass of those galaxies. Optical photographs show spiral galaxies to be graceful pinwheels of billions of stars, with the light falling off steadily from the central to the outer regions. Since the light is produced by stars, we naturally expect the matter and its associated gravitational force field to show a similar concentration. It follows, then, that the speed of rotation of the stars and gas should decline as one moves from the inner to the outer regions of galaxies.
Much to the surprise and consternation of astronomers, this is not what is observed. As radio and optical observations have extended the velocity measurements for the stars and gas to the outer regions of spiral galaxies, they have found that the stars and gas clouds are moving at the same speed as the ones closer in! A substantial part of the mass of the galaxy is not concentrated toward the center of the galaxy but must be [160] distributed in some dark, unseen halo surrounding the visible galaxy. The outer regions of galaxies, faint and inconspicuous on a photograph, may actually contain most of the matter. In the words of astronomers Margaret and Geoffrey Burbidge, it appears that "the tail wags the dog."

Perhaps this heavy element ejection theory explains the missing matter. If all stars produced companion metal stars in their formation, then this explains the perfect balance of galaxial rotation BETTER than the black hole at the center of the galaxy theory.

But we do detect odd wobbles.  The out planet orbits don't quite conform to our laws of gravity. This first allowed us to extract out the notion of a PLUTO planetoid years before confirming its existance. And now again, because it is so far out so vastly far out, it's effect is ever so tiny.

In January 2015, Caltech astronomers Konstantin Batygin and Mike Brown announced new research that provides evidence of a giant planet tracing an unusual, elongated orbit in the outer solar system. The prediction is based on detailed mathematical modeling and computer simulations, not direct observation.

This large object could explain the unique orbits of at least five smaller objects discovered in the distant Kuiper Belt.

"The possibility of a new planet is certainly an exciting one for me as a planetary scientist and for all of us," said Jim Green, director of NASA's Planetary Science Division. "This is not, however, the detection or discovery of a new planet. It's too early to say with certainty there's a so-called Planet X. What we're seeing is an early prediction based on modeling from limited observations. It's the start of a process that could lead to an exciting result."

The Caltech scientists believe Planet X may have has a mass about 10 times that of Earth and be similar in size to Uranus or Neptune. The predicted orbit is about 20 times farther from our sun on average than Neptune (which orbits the sun at an average distance of 2.8 billion miles). It would take this new planet between 10,000 and 20,000 years to make just one full orbit around the sun (where Neptune completes an orbit roughly every 165 years). - NASA

For the moment. Our companion metal sun is dark and leathal.  When it comes crashing in, we see the zodiac precession accelerate dramatically from 1/72 of a degree arc per year to 100 times that. This means it is end times, the days of great flames. Why does this happen? It is literally the mass of Nibiru our metal star steering our entire solar system sideways against the ecliptic. Will we build our pyramids when we see it? Or escape into far out space only to return to re-seed our own people.

Books are selling like HotCakes in Japan!

Ok, truth be told no one ever reads my books, but its still fun to see it. Wakarmasitaka!