As its name means, gradient-based methods need the gradient of objective functions to design variables. The evaluation of gradient can be achieved by finite difference method, linearized method or adjoint method. Both finite difference method and linearized method has a time-cost proportional to the number of design variables and not suitable for design optimization with a large number of design variables. Apart from that, finite difference method has a notorious disadvantage of subtraction cancellation and is not recommended for practical design application.
Suppose a cost function $J$ is defined as follows, where and Finite difference method: