Finding a minimum of quadratic function with quadratic equality constraints.












0












$begingroup$


I need to find the minimum of the function $x^TAx$ with the condition, that $x^TBx = 1$, where both $A$ and $B$ are positive definite (symmetric) matrices.



I put a lagrangian $L = x^TAx - alpha * x^TBx$ and obtained $Ax = alpha * Bx$. From there, I can get $x = alpha * A^{-1}Bx$ and $B^{-1}Ax = alpha * x$, since both $A$ and $B$ are regular.



Combining those two, I get $B^{-1}AB^{-1}Ax = alpha^2 x$. Now, since both $A$ and $B$ were symmetric positive definite, they will have real, positive eigenvalues. Also, from our constraint ($x^TBx = 1$), we obtain $alpha = x^TAx$. Is it then safe to assume, that taking square root of the smallest eigenvalue is the solution to the original problem, with the eigenvector associated (scaled appropriately for the condition to hold) as its argument?



Is there any other way, how to solve this?






linear-algebra optimization







share|cite|improve this question









$endgroup$












  • $begingroup$
    Test your hypothesis in two dimensions with $A=I$ and $B$ diagonal, so that it represents an axis-aligned ellipse. Which eigenvalue corresponds to the shorter principal axis?
    $endgroup$
    – amd
    Dec 4 '18 at 0:17


















0












$begingroup$


I need to find the minimum of the function $x^TAx$ with the condition, that $x^TBx = 1$, where both $A$ and $B$ are positive definite (symmetric) matrices.



I put a lagrangian $L = x^TAx - alpha * x^TBx$ and obtained $Ax = alpha * Bx$. From there, I can get $x = alpha * A^{-1}Bx$ and $B^{-1}Ax = alpha * x$, since both $A$ and $B$ are regular.



Combining those two, I get $B^{-1}AB^{-1}Ax = alpha^2 x$. Now, since both $A$ and $B$ were symmetric positive definite, they will have real, positive eigenvalues. Also, from our constraint ($x^TBx = 1$), we obtain $alpha = x^TAx$. Is it then safe to assume, that taking square root of the smallest eigenvalue is the solution to the original problem, with the eigenvector associated (scaled appropriately for the condition to hold) as its argument?



Is there any other way, how to solve this?






linear-algebra optimization







share|cite|improve this question









$endgroup$












  • $begingroup$
    Test your hypothesis in two dimensions with $A=I$ and $B$ diagonal, so that it represents an axis-aligned ellipse. Which eigenvalue corresponds to the shorter principal axis?
    $endgroup$
    – amd
    Dec 4 '18 at 0:17
















0












0








0





$begingroup$


I need to find the minimum of the function $x^TAx$ with the condition, that $x^TBx = 1$, where both $A$ and $B$ are positive definite (symmetric) matrices.



I put a lagrangian $L = x^TAx - alpha * x^TBx$ and obtained $Ax = alpha * Bx$. From there, I can get $x = alpha * A^{-1}Bx$ and $B^{-1}Ax = alpha * x$, since both $A$ and $B$ are regular.



Combining those two, I get $B^{-1}AB^{-1}Ax = alpha^2 x$. Now, since both $A$ and $B$ were symmetric positive definite, they will have real, positive eigenvalues. Also, from our constraint ($x^TBx = 1$), we obtain $alpha = x^TAx$. Is it then safe to assume, that taking square root of the smallest eigenvalue is the solution to the original problem, with the eigenvector associated (scaled appropriately for the condition to hold) as its argument?



Is there any other way, how to solve this?






linear-algebra optimization







share|cite|improve this question









$endgroup$




I need to find the minimum of the function $x^TAx$ with the condition, that $x^TBx = 1$, where both $A$ and $B$ are positive definite (symmetric) matrices.



I put a lagrangian $L = x^TAx - alpha * x^TBx$ and obtained $Ax = alpha * Bx$. From there, I can get $x = alpha * A^{-1}Bx$ and $B^{-1}Ax = alpha * x$, since both $A$ and $B$ are regular.



Combining those two, I get $B^{-1}AB^{-1}Ax = alpha^2 x$. Now, since both $A$ and $B$ were symmetric positive definite, they will have real, positive eigenvalues. Also, from our constraint ($x^TBx = 1$), we obtain $alpha = x^TAx$. Is it then safe to assume, that taking square root of the smallest eigenvalue is the solution to the original problem, with the eigenvector associated (scaled appropriately for the condition to hold) as its argument?



Is there any other way, how to solve this?






linear-algebra optimization




linear-algebra optimization






share|cite|improve this question













share|cite|improve this question











share|cite|improve this question




share|cite|improve this question










asked Dec 3 '18 at 22:22









Jonáš KulhánekJonáš Kulhánek

729




729












  • $begingroup$
    Test your hypothesis in two dimensions with $A=I$ and $B$ diagonal, so that it represents an axis-aligned ellipse. Which eigenvalue corresponds to the shorter principal axis?
    $endgroup$
    – amd
    Dec 4 '18 at 0:17




















  • $begingroup$
    Test your hypothesis in two dimensions with $A=I$ and $B$ diagonal, so that it represents an axis-aligned ellipse. Which eigenvalue corresponds to the shorter principal axis?
    $endgroup$
    – amd
    Dec 4 '18 at 0:17


















$begingroup$
Test your hypothesis in two dimensions with $A=I$ and $B$ diagonal, so that it represents an axis-aligned ellipse. Which eigenvalue corresponds to the shorter principal axis?
$endgroup$
– amd
Dec 4 '18 at 0:17






$begingroup$
Test your hypothesis in two dimensions with $A=I$ and $B$ diagonal, so that it represents an axis-aligned ellipse. Which eigenvalue corresponds to the shorter principal axis?
$endgroup$
– amd
Dec 4 '18 at 0:17












1 Answer
1






active

oldest

votes


















1












$begingroup$

First of all, the problem is not convex. Non-convexity stems from the equality constraint. You try to solve a Quadratically Constrained Quadratic Program(QCQP). So, although I could not fully understand your argument, the Lagrangian may not help you to find the exact solution, rather it provides a good lower bounds. I will offer a different perspective, consider following problem:
$$min x^TAx, s.t. x^TBx = 1 \ equiv min trace(x^TAx), s.t. trace(x^TBx) = 1 \equiv min trace(Axx^T), s.t. trace(Bxx^T) = 1 \ equiv min trace(AX), s.t. trace(BX) = 1, rank(X) = 1$$
Your original problem is transformed into Semi-definite Program (SDP) which is still non-convex, but you can relax the rank constraint and you will get a convex Relaxed SDP, which is a linear program in your case:
$$ min trace(AX), \s.t. trace(BX) = 1 $$
Nice thing about this relaxation is that the solution of the relaxed SDP is equivalent to original SDP (so your original problem), since you have only one constraint. All of this relaxation procedure and the conditions for optimal solution can be found in the Lou et al's excellent lecture notes[1]. Solving this problem analytically rather than via software may not be possible.



[1]Luo, Zhi-Quan, et al. "Semidefinite relaxation of quadratic optimization problems." IEEE Signal Processing Magazine 27.3 (2010): 20-34.






share|cite|improve this answer









$endgroup$













    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "69"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    noCode: true, onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3024781%2ffinding-a-minimum-of-quadratic-function-with-quadratic-equality-constraints%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    1












    $begingroup$

    First of all, the problem is not convex. Non-convexity stems from the equality constraint. You try to solve a Quadratically Constrained Quadratic Program(QCQP). So, although I could not fully understand your argument, the Lagrangian may not help you to find the exact solution, rather it provides a good lower bounds. I will offer a different perspective, consider following problem:
    $$min x^TAx, s.t. x^TBx = 1 \ equiv min trace(x^TAx), s.t. trace(x^TBx) = 1 \equiv min trace(Axx^T), s.t. trace(Bxx^T) = 1 \ equiv min trace(AX), s.t. trace(BX) = 1, rank(X) = 1$$
    Your original problem is transformed into Semi-definite Program (SDP) which is still non-convex, but you can relax the rank constraint and you will get a convex Relaxed SDP, which is a linear program in your case:
    $$ min trace(AX), \s.t. trace(BX) = 1 $$
    Nice thing about this relaxation is that the solution of the relaxed SDP is equivalent to original SDP (so your original problem), since you have only one constraint. All of this relaxation procedure and the conditions for optimal solution can be found in the Lou et al's excellent lecture notes[1]. Solving this problem analytically rather than via software may not be possible.



    [1]Luo, Zhi-Quan, et al. "Semidefinite relaxation of quadratic optimization problems." IEEE Signal Processing Magazine 27.3 (2010): 20-34.






    share|cite|improve this answer









    $endgroup$


















      1












      $begingroup$

      First of all, the problem is not convex. Non-convexity stems from the equality constraint. You try to solve a Quadratically Constrained Quadratic Program(QCQP). So, although I could not fully understand your argument, the Lagrangian may not help you to find the exact solution, rather it provides a good lower bounds. I will offer a different perspective, consider following problem:
      $$min x^TAx, s.t. x^TBx = 1 \ equiv min trace(x^TAx), s.t. trace(x^TBx) = 1 \equiv min trace(Axx^T), s.t. trace(Bxx^T) = 1 \ equiv min trace(AX), s.t. trace(BX) = 1, rank(X) = 1$$
      Your original problem is transformed into Semi-definite Program (SDP) which is still non-convex, but you can relax the rank constraint and you will get a convex Relaxed SDP, which is a linear program in your case:
      $$ min trace(AX), \s.t. trace(BX) = 1 $$
      Nice thing about this relaxation is that the solution of the relaxed SDP is equivalent to original SDP (so your original problem), since you have only one constraint. All of this relaxation procedure and the conditions for optimal solution can be found in the Lou et al's excellent lecture notes[1]. Solving this problem analytically rather than via software may not be possible.



      [1]Luo, Zhi-Quan, et al. "Semidefinite relaxation of quadratic optimization problems." IEEE Signal Processing Magazine 27.3 (2010): 20-34.






      share|cite|improve this answer









      $endgroup$
















        1












        1








        1





        $begingroup$

        First of all, the problem is not convex. Non-convexity stems from the equality constraint. You try to solve a Quadratically Constrained Quadratic Program(QCQP). So, although I could not fully understand your argument, the Lagrangian may not help you to find the exact solution, rather it provides a good lower bounds. I will offer a different perspective, consider following problem:
        $$min x^TAx, s.t. x^TBx = 1 \ equiv min trace(x^TAx), s.t. trace(x^TBx) = 1 \equiv min trace(Axx^T), s.t. trace(Bxx^T) = 1 \ equiv min trace(AX), s.t. trace(BX) = 1, rank(X) = 1$$
        Your original problem is transformed into Semi-definite Program (SDP) which is still non-convex, but you can relax the rank constraint and you will get a convex Relaxed SDP, which is a linear program in your case:
        $$ min trace(AX), \s.t. trace(BX) = 1 $$
        Nice thing about this relaxation is that the solution of the relaxed SDP is equivalent to original SDP (so your original problem), since you have only one constraint. All of this relaxation procedure and the conditions for optimal solution can be found in the Lou et al's excellent lecture notes[1]. Solving this problem analytically rather than via software may not be possible.



        [1]Luo, Zhi-Quan, et al. "Semidefinite relaxation of quadratic optimization problems." IEEE Signal Processing Magazine 27.3 (2010): 20-34.






        share|cite|improve this answer









        $endgroup$



        First of all, the problem is not convex. Non-convexity stems from the equality constraint. You try to solve a Quadratically Constrained Quadratic Program(QCQP). So, although I could not fully understand your argument, the Lagrangian may not help you to find the exact solution, rather it provides a good lower bounds. I will offer a different perspective, consider following problem:
        $$min x^TAx, s.t. x^TBx = 1 \ equiv min trace(x^TAx), s.t. trace(x^TBx) = 1 \equiv min trace(Axx^T), s.t. trace(Bxx^T) = 1 \ equiv min trace(AX), s.t. trace(BX) = 1, rank(X) = 1$$
        Your original problem is transformed into Semi-definite Program (SDP) which is still non-convex, but you can relax the rank constraint and you will get a convex Relaxed SDP, which is a linear program in your case:
        $$ min trace(AX), \s.t. trace(BX) = 1 $$
        Nice thing about this relaxation is that the solution of the relaxed SDP is equivalent to original SDP (so your original problem), since you have only one constraint. All of this relaxation procedure and the conditions for optimal solution can be found in the Lou et al's excellent lecture notes[1]. Solving this problem analytically rather than via software may not be possible.



        [1]Luo, Zhi-Quan, et al. "Semidefinite relaxation of quadratic optimization problems." IEEE Signal Processing Magazine 27.3 (2010): 20-34.







        share|cite|improve this answer












        share|cite|improve this answer



        share|cite|improve this answer










        answered Dec 4 '18 at 9:51









        KurbanKurban

        405




        405






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Mathematics Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3024781%2ffinding-a-minimum-of-quadratic-function-with-quadratic-equality-constraints%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            Index of /

            This page is only for reference, If you need detailed information, please check here
            Read more

            Tribalistas

            Image
              Nota: Para o álbum homônimo, veja Tribalistas (álbum de 2002). Para o segundo álbum homônimo, veja Tribalistas (álbum de 2017). Para o tipo de organização tribal, veja Tribalismo. Tribalistas Informação geral Origem Salvador e Rio de Janeiro País   Brasil Gênero(s) MPB Pop Samba Período em atividade 2002–2004 2017–atualmente Gravadora(s) EMI Phonomotor Records Universal Music Afiliação(ões) Dadi Pretinho da Serrinha Margareth Menezes Integrantes Arnaldo Antunes Carlinhos Brown Marisa Monte Página oficial www2.uol.com.br/tribalistas/ ...
            Read more

            Listed building

            Image
            Listed building (literalmente: prédio listado ) é toda estrutura protegida por lei pelo complexo sistema de tombamento do Reino Unido. Um listed building é qualquer estrutura oficialmente designada como tendo especial interesse arquitetônico, histórico ou cultural. É um estatuto amplamente usado, aplicado a centenas de milhares de estruturas. Estruturas tombadas não podem ser alteradas sem permissão especial, e desfrutam de isenções fiscais e outros benefícios. Cada um dos países do Reino Unido trata tombamento de forma diferente: Índice 1 Inglaterra 2 País de Gales 3 Irlanda do Norte 4 Escócia 5 Referências Inglaterra | O Palácio de Buckingham, Londres. As estruturas tombadas são listadas no Statutory List of Buildings of Special Architectural or Historic Interest , e incluem campos, sinais de trânsito, piers e áreas geográficas. A principal legislação pertinente é o Ancient Monuments and Archaeological Areas Act 1979 , [ 1 ] embora legisla...
            Read more