Yes, I believe they should be treated as records within the QMS and must be maintained in the Supplier File. You don't necessarily have to show it during an audit but it's best to be kept in file in case asked for.
Example: (I'm assuming the agreement consists of requirements you intend the supplier to follow to meet the desired quality of products or services they supply to you)
The Supplier Quality Agreement states that the supplier must maintain records for 5 years (retention period).
Now this, in itself isn't sufficient to demonstrate the supplier meets this requirement, however it is a management system requirement from you to your suppliers and is a document (or record) of you providing your suppliers the requirement.
I Quote ISO 9001:2015 Clause 8.4.1
"The organization shall determine the controls to be applied to externally provided processes, products
and services when:
a) products and services from external providers are intended for incorporation into the organization’s
own products and services;
b) products and services are provided directly to the customer(s) by external providers on behalf of
the organization;
c) a process, or part of a process, is provided by an external provider as a result of a decision by the
organization."
Hope this clarifies