This is because we have different requirements to an API user and an API developer.
A developer is responsible for providing a precise specification and a correct implementation. Therefore if the developer is already aware that an argument is optional the implementation must deal with it correctly, whether it being a null or an Optional. The API should be as simple as possible to the user, and null is the simplest.
On the other hand, the result is passed from the API developer to the user. However the specification is complete and verbose, there is still a chance that the user is either unaware of it or just lazy to deal with it. In this case, the Optional result forces the user to write some extra code to deal with a possible empty result.