A helpful warning when adding sparsity constraints to NMF

Describe the issue linked to the documentation

Currently the documentation of NMF, as well as extensions like the MiniBatchNMF provide useful comments and warnings for beginners. For example, what type of initialization is suited based on whether sparsity is desired etc.

One thing that is not however addressed is the scale ambiguity of solving NMF. Specifically, if one desires sparsity on one of the matrices, say W, one should make sure the norm of the other matrix, in this case H, is controlled. Otherwise, a trivial solution would be a rescaled version of W and H, where the norm of W is decreased and the norm of H is increased. This would give the same exact (dot-product) output, while reducing both the L1 and L2 norm of the W matrix. If the user doesn't manually inspect the norms of H later, they may be mislead on what is actually happening. They may think that they have a more sparse factorization, whereas for the most part, they have arrived at a similar solution, just that the matrices have been rescaled. This would really hinder the actual sparsity of the factorization. You can also find this issue discussed in the last paragraph on the first page of this paper.

Suggest a potential alternative/fix

I checked the literature and people often choose between doing Projected Gradient Descent (i.e to project the other matrix to a specified norm so that the model doesn't cheat) or doing a norm regularization on the other matrix as well. Since adding PGD would be too much of a change, I think letting the user know and maybe encouraging them to also add a sparsity constraint on the other matrix is the way to go.

I think there should be a simple warning when either one of alpha_w or alpha_h is enabled while the other is zero. It would simply warn the user about the scale ambiguity of the NMF. Otherwise, I think having it as part of the documentation would also work. But as of now, it can be very misleading, especially for people new to the NMF.

If you agree, I would love to make a pull request on this, as it would also be my first PR to a large project :)