Substack is better

The difference between a project that feeds semi-retired Googlers and a product by a startup that needs it to succeed is pretty phenomenal.

Anyway, here's my first serious post there: https://hnfong.substack.com/p/discrete-is-harder 

Since I paused reading the article that inspired my quote after reading said quote, here's a bit of extra stuff that didn't make it into the article.

Quote:

---

In general, the most striking difference between deep learning and classical algorithmics is how declaratively deep learning researchers think. They think hard about what the right loss function to optimize is or which part of the net to keep fixed and which part to optimize during an experiment. But they think less about how to actually achieve the goal of minimizing the loss function. This is often done by including a few tiny lines in the code, like network.train() or network.backward(). To me, the essence of deep learning has nothing to do with trying to mimic biological systems or something like that; it’s the observation that if your circuits are continuous, there’s a clear algorithmic way of inverting/optimizing them using backpropagation.

From the perspective of someone used to algorithms like Dijkstra’s algorithm, quicksort, and so on, this declarative approach of thinking in terms of loss functions and architectures, rather than how the net is actually optimized, sounds very alien. But this is how the whole algorithmic world would look like if P equaled NP! In that world, we’d all program declaratively in Prolog and use some kind of .solve() function at the end that would internally run a fast SAT solver to solve the problem defined by our declarations.

---

Very shortly summarized, for continuous functions, there's a general way of finding what you need by declaratively saying what you want, and let the universe do the work for you.

Is the universe continuous? That's always the ultimate question.

-------

Oh and I forgot. One very curious way of implementing the rumored "5th dimension ascension" is through adoption of continuous thought and technologies and discarding boolean ones.  That's a very sobering thought (to me at least). "Making the world more continuous and less discrete" sounds like a manifesto written by a drunk and dissociated meta-mathematician ... but I mean, if you think about it...