What is asymmetric relaxation backpropagation? Ask Question

Asked 5 years ago Modified today Viewed 256 times

4 $\begingroup$ In Chapter 8 , section 8.5.2, Raul Rojas describes how the weights for a layer of a neural network can be calculated using a pseudoinverse of the sigmoid function in the nodes, he explains this is an example of symmetric relaxation. But the chapter doesn't explain what asymmetric relaxation would be or how it is done. So, what is asymmetric relaxation and how would it be done in a simple neural network using a sigmoid function in its nodes? neural-networks terminology backpropagation Share Improve this question Follow edited Dec 30, 2020 at 19:07 asked Dec 19, 2020 at 22:53 EmmanuelMess 247 3 3 silver badges 16 16 bronze badges $\endgroup$ Add a comment | 1 Answer 1 Sorted by: Reset to default Highest score (default) Date modified (newest first) Date created (oldest first) 0 $\begingroup$ I'll give you my initial $0.02 for symmetric relaxation or relaxation in general in working with neural networks. The book covers 'Weight perturbation' and this is a basic outline of that. Say you want to host a wedding and every person gives you a 'must-have' list of requirements for them to attend. You can abide by all the requirements of each wedding guest or start 'uninviting' guests whose restrictions cause too many complications. There are several kinds of relaxation. I've only used Lagrangian relaxation, so my experience is biased to that application. Think of it like this: you are traveling from New York to LA and you want to optimize for time, if you 'relax' the constraints, you can just fly instead of driving. This, however, creates an increased cost of the air ticket. By relaxing the constraints you remove the isolating requirement that you must travel by car. Symmetric relaxation can be a challenging subject, so I'll include a few links academic research Academic research arxiv.org is another site I use for research. Hope this helps. I also found a link on Medium which is another good source for application, theory, and implementation of algorithms. Medium Lagrangian Relaxation Share Improve this answer Follow edited Dec 28, 2020 at 22:02 nbro 43.2k 14 14 gold badges 121 121 silver badges 222 222 bronze badges answered Dec 28, 2020 at 21:09 CoffeeBaconAddict 81 7 7 bronze badges $\endgroup$ 1 2 $\begingroup$ Literally just realized that I had a typo in the final question, the question is about asymmetric relaxation, not symmetric. $\endgroup$ EmmanuelMess – EmmanuelMess 2020-12-30 19:08:33 +00:00 Commented Dec 30, 2020 at 19:08 Add a comment | You must log in to answer this question. Start asking to get answers Find the answer to your question by asking. Ask question Explore related questions neural-networks terminology backpropagation See similar questions with these tags.

4 $\begingroup$ In Chapter 8 , section 8.5.2, Raul Rojas describes how the weights for a layer of a neural network can be calculated using a pseudoinverse of the sigmoid function in the nodes, he explains this is an example of symmetric relaxation. But the chapter doesn't explain what asymmetric relaxation would be or how it is done. So, what is asymmetric relaxation and how would it be done in a simple neural network using a sigmoid function in its nodes? neural-networks terminology backpropagation Share Improve this question Follow edited Dec 30, 2020 at 19:07 asked Dec 19, 2020 at 22:53 EmmanuelMess 247 3 3 silver badges 16 16 bronze badges $\endgroup$ Add a comment |

4 $\begingroup$ In Chapter 8 , section 8.5.2, Raul Rojas describes how the weights for a layer of a neural network can be calculated using a pseudoinverse of the sigmoid function in the nodes, he explains this is an example of symmetric relaxation. But the chapter doesn't explain what asymmetric relaxation would be or how it is done. So, what is asymmetric relaxation and how would it be done in a simple neural network using a sigmoid function in its nodes? neural-networks terminology backpropagation Share Improve this question Follow edited Dec 30, 2020 at 19:07 asked Dec 19, 2020 at 22:53 EmmanuelMess 247 3 3 silver badges 16 16 bronze badges $\endgroup$ Add a comment |

$\begingroup$ In Chapter 8 , section 8.5.2, Raul Rojas describes how the weights for a layer of a neural network can be calculated using a pseudoinverse of the sigmoid function in the nodes, he explains this is an example of symmetric relaxation. But the chapter doesn't explain what asymmetric relaxation would be or how it is done. So, what is asymmetric relaxation and how would it be done in a simple neural network using a sigmoid function in its nodes? neural-networks terminology backpropagation Share Improve this question Follow edited Dec 30, 2020 at 19:07 asked Dec 19, 2020 at 22:53 EmmanuelMess 247 3 3 silver badges 16 16 bronze badges $\endgroup$

In Chapter 8 , section 8.5.2, Raul Rojas describes how the weights for a layer of a neural network can be calculated using a pseudoinverse of the sigmoid function in the nodes, he explains this is an example of symmetric relaxation. But the chapter doesn't explain what asymmetric relaxation would be or how it is done. So, what is asymmetric relaxation and how would it be done in a simple neural network using a sigmoid function in its nodes?

In Chapter 8 , section 8.5.2, Raul Rojas describes how the weights for a layer of a neural network can be calculated using a pseudoinverse of the sigmoid function in the nodes, he explains this is an example of symmetric relaxation.

But the chapter doesn't explain what asymmetric relaxation would be or how it is done.

So, what is asymmetric relaxation and how would it be done in a simple neural network using a sigmoid function in its nodes?

neural-networks terminology backpropagation

neural-networks terminology backpropagation

neural-networks terminology backpropagation

Share Improve this question Follow edited Dec 30, 2020 at 19:07 asked Dec 19, 2020 at 22:53 EmmanuelMess 247 3 3 silver badges 16 16 bronze badges

Share Improve this question Follow edited Dec 30, 2020 at 19:07 asked Dec 19, 2020 at 22:53 EmmanuelMess 247 3 3 silver badges 16 16 bronze badges

Share Improve this question Follow

Share Improve this question Follow

Share Improve this question Follow

Improve this question

edited Dec 30, 2020 at 19:07

edited Dec 30, 2020 at 19:07

edited Dec 30, 2020 at 19:07

edited Dec 30, 2020 at 19:07

asked Dec 19, 2020 at 22:53 EmmanuelMess 247 3 3 silver badges 16 16 bronze badges

asked Dec 19, 2020 at 22:53 EmmanuelMess 247 3 3 silver badges 16 16 bronze badges

asked Dec 19, 2020 at 22:53

asked Dec 19, 2020 at 22:53

EmmanuelMess 247 3 3 silver badges 16 16 bronze badges

247 3 3 silver badges 16 16 bronze badges

1 Answer 1 Sorted by: Reset to default Highest score (default) Date modified (newest first) Date created (oldest first) 0 $\begingroup$ I'll give you my initial $0.02 for symmetric relaxation or relaxation in general in working with neural networks. The book covers 'Weight perturbation' and this is a basic outline of that. Say you want to host a wedding and every person gives you a 'must-have' list of requirements for them to attend. You can abide by all the requirements of each wedding guest or start 'uninviting' guests whose restrictions cause too many complications. There are several kinds of relaxation. I've only used Lagrangian relaxation, so my experience is biased to that application. Think of it like this: you are traveling from New York to LA and you want to optimize for time, if you 'relax' the constraints, you can just fly instead of driving. This, however, creates an increased cost of the air ticket. By relaxing the constraints you remove the isolating requirement that you must travel by car. Symmetric relaxation can be a challenging subject, so I'll include a few links academic research Academic research arxiv.org is another site I use for research. Hope this helps. I also found a link on Medium which is another good source for application, theory, and implementation of algorithms. Medium Lagrangian Relaxation Share Improve this answer Follow edited Dec 28, 2020 at 22:02 nbro 43.2k 14 14 gold badges 121 121 silver badges 222 222 bronze badges answered Dec 28, 2020 at 21:09 CoffeeBaconAddict 81 7 7 bronze badges $\endgroup$ 1 2 $\begingroup$ Literally just realized that I had a typo in the final question, the question is about asymmetric relaxation, not symmetric. $\endgroup$ EmmanuelMess – EmmanuelMess 2020-12-30 19:08:33 +00:00 Commented Dec 30, 2020 at 19:08 Add a comment | You must log in to answer this question. Start asking to get answers Find the answer to your question by asking. Ask question Explore related questions neural-networks terminology backpropagation See similar questions with these tags.

1 Answer 1 Sorted by: Reset to default Highest score (default) Date modified (newest first) Date created (oldest first)

1 Answer 1 Sorted by: Reset to default Highest score (default) Date modified (newest first) Date created (oldest first)

Sorted by: Reset to default Highest score (default) Date modified (newest first) Date created (oldest first)

Sorted by: Reset to default Highest score (default) Date modified (newest first) Date created (oldest first)

Sorted by: Reset to default

Highest score (default) Date modified (newest first) Date created (oldest first)

0 $\begingroup$ I'll give you my initial $0.02 for symmetric relaxation or relaxation in general in working with neural networks. The book covers 'Weight perturbation' and this is a basic outline of that. Say you want to host a wedding and every person gives you a 'must-have' list of requirements for them to attend. You can abide by all the requirements of each wedding guest or start 'uninviting' guests whose restrictions cause too many complications. There are several kinds of relaxation. I've only used Lagrangian relaxation, so my experience is biased to that application. Think of it like this: you are traveling from New York to LA and you want to optimize for time, if you 'relax' the constraints, you can just fly instead of driving. This, however, creates an increased cost of the air ticket. By relaxing the constraints you remove the isolating requirement that you must travel by car. Symmetric relaxation can be a challenging subject, so I'll include a few links academic research Academic research arxiv.org is another site I use for research. Hope this helps. I also found a link on Medium which is another good source for application, theory, and implementation of algorithms. Medium Lagrangian Relaxation Share Improve this answer Follow edited Dec 28, 2020 at 22:02 nbro 43.2k 14 14 gold badges 121 121 silver badges 222 222 bronze badges answered Dec 28, 2020 at 21:09 CoffeeBaconAddict 81 7 7 bronze badges $\endgroup$ 1 2 $\begingroup$ Literally just realized that I had a typo in the final question, the question is about asymmetric relaxation, not symmetric. $\endgroup$ EmmanuelMess – EmmanuelMess 2020-12-30 19:08:33 +00:00 Commented Dec 30, 2020 at 19:08 Add a comment |

0 $\begingroup$ I'll give you my initial $0.02 for symmetric relaxation or relaxation in general in working with neural networks. The book covers 'Weight perturbation' and this is a basic outline of that. Say you want to host a wedding and every person gives you a 'must-have' list of requirements for them to attend. You can abide by all the requirements of each wedding guest or start 'uninviting' guests whose restrictions cause too many complications. There are several kinds of relaxation. I've only used Lagrangian relaxation, so my experience is biased to that application. Think of it like this: you are traveling from New York to LA and you want to optimize for time, if you 'relax' the constraints, you can just fly instead of driving. This, however, creates an increased cost of the air ticket. By relaxing the constraints you remove the isolating requirement that you must travel by car. Symmetric relaxation can be a challenging subject, so I'll include a few links academic research Academic research arxiv.org is another site I use for research. Hope this helps. I also found a link on Medium which is another good source for application, theory, and implementation of algorithms. Medium Lagrangian Relaxation Share Improve this answer Follow edited Dec 28, 2020 at 22:02 nbro 43.2k 14 14 gold badges 121 121 silver badges 222 222 bronze badges answered Dec 28, 2020 at 21:09 CoffeeBaconAddict 81 7 7 bronze badges $\endgroup$ 1 2 $\begingroup$ Literally just realized that I had a typo in the final question, the question is about asymmetric relaxation, not symmetric. $\endgroup$ EmmanuelMess – EmmanuelMess 2020-12-30 19:08:33 +00:00 Commented Dec 30, 2020 at 19:08 Add a comment |

$\begingroup$ I'll give you my initial $0.02 for symmetric relaxation or relaxation in general in working with neural networks. The book covers 'Weight perturbation' and this is a basic outline of that. Say you want to host a wedding and every person gives you a 'must-have' list of requirements for them to attend. You can abide by all the requirements of each wedding guest or start 'uninviting' guests whose restrictions cause too many complications. There are several kinds of relaxation. I've only used Lagrangian relaxation, so my experience is biased to that application. Think of it like this: you are traveling from New York to LA and you want to optimize for time, if you 'relax' the constraints, you can just fly instead of driving. This, however, creates an increased cost of the air ticket. By relaxing the constraints you remove the isolating requirement that you must travel by car. Symmetric relaxation can be a challenging subject, so I'll include a few links academic research Academic research arxiv.org is another site I use for research. Hope this helps. I also found a link on Medium which is another good source for application, theory, and implementation of algorithms. Medium Lagrangian Relaxation Share Improve this answer Follow edited Dec 28, 2020 at 22:02 nbro 43.2k 14 14 gold badges 121 121 silver badges 222 222 bronze badges answered Dec 28, 2020 at 21:09 CoffeeBaconAddict 81 7 7 bronze badges $\endgroup$

I'll give you my initial $0.02 for symmetric relaxation or relaxation in general in working with neural networks. The book covers 'Weight perturbation' and this is a basic outline of that. Say you want to host a wedding and every person gives you a 'must-have' list of requirements for them to attend. You can abide by all the requirements of each wedding guest or start 'uninviting' guests whose restrictions cause too many complications. There are several kinds of relaxation. I've only used Lagrangian relaxation, so my experience is biased to that application. Think of it like this: you are traveling from New York to LA and you want to optimize for time, if you 'relax' the constraints, you can just fly instead of driving. This, however, creates an increased cost of the air ticket. By relaxing the constraints you remove the isolating requirement that you must travel by car. Symmetric relaxation can be a challenging subject, so I'll include a few links academic research Academic research arxiv.org is another site I use for research. Hope this helps. I also found a link on Medium which is another good source for application, theory, and implementation of algorithms. Medium Lagrangian Relaxation

I'll give you my initial $0.02 for symmetric relaxation or relaxation in general in working with neural networks. The book covers 'Weight perturbation' and this is a basic outline of that. Say you want to host a wedding and every person gives you a 'must-have' list of requirements for them to attend. You can abide by all the requirements of each wedding guest or start 'uninviting' guests whose restrictions cause too many complications.

There are several kinds of relaxation. I've only used Lagrangian relaxation, so my experience is biased to that application. Think of it like this: you are traveling from New York to LA and you want to optimize for time, if you 'relax' the constraints, you can just fly instead of driving. This, however, creates an increased cost of the air ticket. By relaxing the constraints you remove the isolating requirement that you must travel by car.

Symmetric relaxation can be a challenging subject, so I'll include a few links academic research

Academic research arxiv.org is another site I use for research. Hope this helps.

I also found a link on Medium which is another good source for application, theory, and implementation of algorithms. Medium Lagrangian Relaxation

Share Improve this answer Follow edited Dec 28, 2020 at 22:02 nbro 43.2k 14 14 gold badges 121 121 silver badges 222 222 bronze badges answered Dec 28, 2020 at 21:09 CoffeeBaconAddict 81 7 7 bronze badges

Share Improve this answer Follow edited Dec 28, 2020 at 22:02 nbro 43.2k 14 14 gold badges 121 121 silver badges 222 222 bronze badges answered Dec 28, 2020 at 21:09 CoffeeBaconAddict 81 7 7 bronze badges

Share Improve this answer Follow

Share Improve this answer Follow

Share Improve this answer Follow

edited Dec 28, 2020 at 22:02 nbro 43.2k 14 14 gold badges 121 121 silver badges 222 222 bronze badges

edited Dec 28, 2020 at 22:02 nbro 43.2k 14 14 gold badges 121 121 silver badges 222 222 bronze badges

edited Dec 28, 2020 at 22:02

edited Dec 28, 2020 at 22:02

nbro 43.2k 14 14 gold badges 121 121 silver badges 222 222 bronze badges

43.2k 14 14 gold badges 121 121 silver badges 222 222 bronze badges

answered Dec 28, 2020 at 21:09 CoffeeBaconAddict 81 7 7 bronze badges

answered Dec 28, 2020 at 21:09 CoffeeBaconAddict 81 7 7 bronze badges

answered Dec 28, 2020 at 21:09

answered Dec 28, 2020 at 21:09

CoffeeBaconAddict 81 7 7 bronze badges

2 $\begingroup$ Literally just realized that I had a typo in the final question, the question is about asymmetric relaxation, not symmetric. $\endgroup$ EmmanuelMess – EmmanuelMess 2020-12-30 19:08:33 +00:00 Commented Dec 30, 2020 at 19:08 Add a comment |

2 $\begingroup$ Literally just realized that I had a typo in the final question, the question is about asymmetric relaxation, not symmetric. $\endgroup$ EmmanuelMess – EmmanuelMess 2020-12-30 19:08:33 +00:00 Commented Dec 30, 2020 at 19:08

$\begingroup$ Literally just realized that I had a typo in the final question, the question is about asymmetric relaxation, not symmetric. $\endgroup$ EmmanuelMess – EmmanuelMess 2020-12-30 19:08:33 +00:00 Commented Dec 30, 2020 at 19:08

$\begingroup$ Literally just realized that I had a typo in the final question, the question is about asymmetric relaxation, not symmetric. $\endgroup$ EmmanuelMess – EmmanuelMess 2020-12-30 19:08:33 +00:00 Commented Dec 30, 2020 at 19:08

EmmanuelMess – EmmanuelMess

2020-12-30 19:08:33 +00:00

Start asking to get answers Find the answer to your question by asking. Ask question Explore related questions neural-networks terminology backpropagation See similar questions with these tags.

Start asking to get answers Find the answer to your question by asking. Ask question

Start asking to get answers Find the answer to your question by asking. Ask question

Start asking to get answers

Find the answer to your question by asking.

Explore related questions neural-networks terminology backpropagation See similar questions with these tags.

Explore related questions neural-networks terminology backpropagation See similar questions with these tags.

Explore related questions

neural-networks terminology backpropagation

See similar questions with these tags.

The Overflow Blog A new era of Stack Overflow Featured on Meta Native Ads coming soon to Stack Overflow and Stack Exchange A proposal for bringing back Community Promotion & Open Source Ads Related 2 How do I implement softmax forward propagation and backpropagation to replace sigmoid in a neural network? 5 Are on-line backpropagation iterations perpendicular to the constraint? 1 Simple three layer neural network with backpropagation is not approximating tanh function 5 Why is second-order backpropagation useful? 0 What is the correct formula for updating the weights in a 1-single hidden layer neural network? 2 How can the input order of pairs into a neural network not matter (i.e. symmetry)? 1 Do neural networks, trained with backpropagation algorithm, exploit the concept of synaptic plasticity? Hot Network Questions Fastest way to make thousands of files Volume of an isosceles tetrahedron and its derivation What type of tiled ceiling is this? Expected radius of the smallest concentric circle having 2 points inside Tefillah toward Jerusalem or specifically eastward? How does the lack of costs-shifting in USA work? Why is mathematics able to produce such persuasive, rigorous proofs? The column includes other values ​from the same column что сказать (as an idiom or introductory phrase) Current source refuses to maintain set current Was Islamic law (Sharia) ever historically interpreted or modified to permit bacha bazi–like practices? Maximal number of vertices of simplex intersected with linear subspace why is grout cracking while being sealed? Looking for Ubuntu repo for re-installing Jammy update-manager how do we actually reason/think/comprehend about nothing? How do copyleft licenses handle support routines in compilers? What is an example of locally finitely presented algebra which is not finitely presented? сливной meaning in context How to route PCB LED indicators to the outside of an enclosure? Filetype detection doesn't work in vimrc Why is analog pre-amplification strictly required for biopotentials instead of direct high-resolution ADC sampling? Is next-token prediction sufficient to explain emergent capabilities like complex code generation in LLMs? What was man (Adam) guarding Eden from? A "shared lock" implementation more hot questions Question feed

The Overflow Blog A new era of Stack Overflow Featured on Meta Native Ads coming soon to Stack Overflow and Stack Exchange A proposal for bringing back Community Promotion & Open Source Ads

A new era of Stack Overflow

Native Ads coming soon to Stack Overflow and Stack Exchange

A proposal for bringing back Community Promotion & Open Source Ads

Related 2 How do I implement softmax forward propagation and backpropagation to replace sigmoid in a neural network? 5 Are on-line backpropagation iterations perpendicular to the constraint? 1 Simple three layer neural network with backpropagation is not approximating tanh function 5 Why is second-order backpropagation useful? 0 What is the correct formula for updating the weights in a 1-single hidden layer neural network? 2 How can the input order of pairs into a neural network not matter (i.e. symmetry)? 1 Do neural networks, trained with backpropagation algorithm, exploit the concept of synaptic plasticity?

2 How do I implement softmax forward propagation and backpropagation to replace sigmoid in a neural network? 5 Are on-line backpropagation iterations perpendicular to the constraint? 1 Simple three layer neural network with backpropagation is not approximating tanh function 5 Why is second-order backpropagation useful? 0 What is the correct formula for updating the weights in a 1-single hidden layer neural network? 2 How can the input order of pairs into a neural network not matter (i.e. symmetry)? 1 Do neural networks, trained with backpropagation algorithm, exploit the concept of synaptic plasticity?

2 How do I implement softmax forward propagation and backpropagation to replace sigmoid in a neural network?

5 Are on-line backpropagation iterations perpendicular to the constraint?

1 Simple three layer neural network with backpropagation is not approximating tanh function

5 Why is second-order backpropagation useful?

0 What is the correct formula for updating the weights in a 1-single hidden layer neural network?

2 How can the input order of pairs into a neural network not matter (i.e. symmetry)?

1 Do neural networks, trained with backpropagation algorithm, exploit the concept of synaptic plasticity?

Hot Network Questions Fastest way to make thousands of files Volume of an isosceles tetrahedron and its derivation What type of tiled ceiling is this? Expected radius of the smallest concentric circle having 2 points inside Tefillah toward Jerusalem or specifically eastward? How does the lack of costs-shifting in USA work? Why is mathematics able to produce such persuasive, rigorous proofs? The column includes other values ​from the same column что сказать (as an idiom or introductory phrase) Current source refuses to maintain set current Was Islamic law (Sharia) ever historically interpreted or modified to permit bacha bazi–like practices? Maximal number of vertices of simplex intersected with linear subspace why is grout cracking while being sealed? Looking for Ubuntu repo for re-installing Jammy update-manager how do we actually reason/think/comprehend about nothing? How do copyleft licenses handle support routines in compilers? What is an example of locally finitely presented algebra which is not finitely presented? сливной meaning in context How to route PCB LED indicators to the outside of an enclosure? Filetype detection doesn't work in vimrc Why is analog pre-amplification strictly required for biopotentials instead of direct high-resolution ADC sampling? Is next-token prediction sufficient to explain emergent capabilities like complex code generation in LLMs? What was man (Adam) guarding Eden from? A "shared lock" implementation more hot questions