Skip to content

The Hessian of x-> dot(x,H,x) should be H+H', not zero #551

Closed

Description

@wmkoolen

Hi all,

ForwardDiff incorrectly computes the Hessian of three-argument-dot.

Cook up some (any) square matrix
julia> H = [1 2 3; 4 5 6; 7 8 9];

Compute Hessian of three-argument dot. Yields all zeros. Should yield H+H'.
julia> ForwardDiff.hessian(x->dot(x,H,x), zeros(3))
3×3 Array{Float64,2}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0

Manually expand that three argument dot call. Now result is the expected H+H':
julia> ForwardDiff.hessian(x->x'H*x, zeros(3))
3×3 Array{Float64,2}:
2.0 6.0 10.0
6.0 10.0 14.0
10.0 14.0 18.0

Curious!

Cheers,

Wouter

Activity

  1. mcabbott commented on Oct 22, 2021

    @mcabbott
    Member

    Notice that if you give slightly nonzero arguments, you get different results:

    julia> H = [1 2 3; 4 5 6; 7 8 9];
    
    julia> ForwardDiff.hessian(x->dot(x,H,x), zeros(3))
    3×3 Matrix{Float64}:
     0.0  0.0  0.0
     0.0  0.0  0.0
     0.0  0.0  0.0
    
    julia> ForwardDiff.hessian(x->dot(x,H,x), [0,0,0.0001])
    3×3 Matrix{Float64}:
     0.0  0.0   3.0
     0.0  0.0   6.0
     3.0  6.0  18.0
    
    julia> ForwardDiff.hessian(x->dot(x,H,x), fill(0.00001, 3))
    3×3 Matrix{Float64}:
      2.0   6.0  10.0
      6.0  10.0  14.0
     10.0  14.0  18.0
    

    So dot must be taking shortcuts when it knows x is zero. This is fixed by #481.

  2. added a commit that references this issue on Nov 9, 2021
  3. added a commit that references this issue on May 16, 2022
  4. added a commit that references this issue on May 31, 2022
  5. added a commit that references this issue on Aug 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions