So I have this little thing where I have a list of numbers, my input vector and multiply it by a matrix, my weights, to get a second list of numbers, my output vector. Now I have a little problem, where I now have an output vector and need to compute which input vector gets the closest to satisfying it. My first thought was to sweep the matrix, but that doesn't work if there's no perfect answer. So I'm a bit confused rn. Probably just forgot the most basic solution to my problem tho.

Edit: I just realised inversed matrixes exist, but kinda forgot how to compute that
Edit: I just took a look at inverse matrixes and turns out they have the same problem so im confused again.

I don't know your use case but the dot product and Euclidean distance can be used to compare vectors. Both produce 1-dimensional numbers so are able to be ordered and have the smallest one chosen.

I assume this is a situation with more linear equations than parameters, so you are forced to find a best fit solution. In that case you can use the matrix normal equation to get the least squares solution.

Assuming your original formula is Ax = b where x is unknown, applying the normal equation would get

A'Ax = A'b –> x = (A'A)^-1*A'b

Where A' is the transposition of A. I can explain how the normal equation works if it is unclear.

Apparently getting the inverse of a matrix is numerically unstable, i.e. floating point inaccuracies compound quickly so sometimes decomposition is used to reduce the effects, though if your case is simple enough it's maybe not an issue.

g6g6g66g6g wrote:

Apparently getting the inverse of a matrix is numerically unstable, i.e. floating point inaccuracies compound quickly so sometimes decomposition is used to reduce the effects, though if your case is simple enough it's maybe not an issue.

I'm trying to make scuffed backprop without bothering to read how real backprop is done.
Anyway, I got most of that formula done, but how do I compute the inverse? I have no idea how to make Gauss-Jordan without hardcoding it for each matrix size and that does seem like the kind of thing where you have a billion edge cases to check for.

Gauss-Jordan can be described as following these steps:

Initialize the inverse matrix.
For each i from 1 to width:

1. Find the largest absolute value in column i checking only the rows including and after i. Swap row i with that row (every action that you do is also done in the inverse matrix). If the largest absolute value is 0 or very close to it then the matrix is non-invertible.
   
2. Divide row i by the value in cell [i, i] (indexes start at 1) to normalize that row.
   
3. For every other row, subtract the multiple of row i that is necessary to zero out column i in that row.
   

Processing and swapping entire rows is done by just iterating over every item in the row. It's all just iteration so it's scalable without hardcoding. The reason why you should swap to the row with the highest value at column i is that that value is what you will be dividing the entire row by. If the value is e.g. <1, then dividing by it will actually magnify everything, including any error, so the largest number is used for the opposite effect. If anything is unclear just let me know.

DEMO

dem_bot wrote:

I'm trying to make scuffed backprop

You probably already know this but to be pedantic neural nets almost by definition can't be linear regression since they are designed not to have the same limitations via the activation function, and as such can't be solved as simply. Typical backpropagation approaches the best values in multiple steps via some calculus i'm not familiar with yet.

awesome-llama wrote:

I don't know your use case but the dot product and Euclidean distance can be used to compare vectors. Both produce 1-dimensional numbers so are able to be ordered and have the smallest one chosen.

What's the picture for
Sorry I'm new to forums

squishyteddie wrote:

What's the picture for
Sorry I'm new to forums

the picture is probably the signature for the post. there is a line that differentiates the signature from the post

squishyteddie wrote:

What's the picture for
Sorry I'm new to forums

everything below the line is signature and appears under every post. In the main screen of the forums at the bottom of the page you can make your own.