Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

investigate TLS13 on MacOS #1979

Open
wfurt opened this issue Jan 21, 2020 · 35 comments
Open

investigate TLS13 on MacOS #1979

wfurt opened this issue Jan 21, 2020 · 35 comments
Assignees
Milestone

Comments

@wfurt
Copy link
Member

wfurt commented Jan 21, 2020

It seems like on Catalina TLS1.3 works out of the box - at least with Safari.
However, when I try to use in .NET I get the following error:

System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
 ---> Interop+AppleCrypto+SslException: bad protocol version
   --- End of inner exception stack trace ---

This may be because API we currently use is marked as obsolete or there may be something missing since TLS1.3 support got out before Catalina. And all TLS1.3 tests are disabled on all versions of MacOS.

related to #1720

@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added the untriaged New issue has not been triaged by the area owner label Jan 21, 2020
@karelz karelz added bug and removed untriaged New issue has not been triaged by the area owner labels Feb 13, 2020
@karelz karelz added this to the 5.0 milestone Feb 13, 2020
@wfurt
Copy link
Member Author

wfurt commented May 26, 2020

It seems like we will need to rewrite PAL to use new Network framework API.
That should fix iOS as well.

https://mailarchive.ietf.org/arch/msg/tls/5QjzTilqjomSyzENtgfaAqQOhbA/

@karelz karelz modified the milestones: 5.0.0, Future Jun 18, 2020
@karelz
Copy link
Member

karelz commented Jun 18, 2020

Triage: Will require PAL rewrite on Mac -- out of scope of 5.0.
The new Catalina API will help also iOS and light up TLS 1.3 there.

@chucker
Copy link

chucker commented Mar 24, 2022

Is this still current? In a .NET 6 console app with macOS 12.3 against a host that only seems to support TLS 1.3, I still get the same stacktrace as above. .NET info:

.NET SDK (reflecting any global.json): Version: 6.0.201 Commit: ef40e6aa06

Runtime Environment:
OS Name: Mac OS X
OS Version: 12.3
OS Platform: Darwin
RID: osx.12-x64
Base Path: /usr/local/share/dotnet/sdk/6.0.201/

Host (useful for support):
Version: 6.0.3
Commit: c24d9a9

.NET SDKs installed:
6.0.201 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 6.0.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

@wfurt
Copy link
Member Author

wfurt commented Mar 24, 2022

yes, it is @chucker. #27727 has more detail but it is essentially same work. It is not clear at this moment if it is even feasible.

@bhoffman-splunk
Copy link

Not to add pressure, but I am more and more encountering TLS 1.3-only systems. This is especially true for internal, greenfield APIs that companies build, since they can "start secure" from the beginning. Any work around (such leveraging OpenSSL as is done in Linux) would be tremendously helpful.

@wfurt
Copy link
Member Author

wfurt commented May 5, 2022

Is there need for client, server or both? I was thinking about some fallback because to allow HTTP2 for macOS developers (#27727) We generally do that for HTTP3/Quic (preview in 6.0) so we have some parts already in a place. if we go down this path, it would probably be opt-in feature with dependency on OpenSSL from Brew/MacPorts. Would that be acceptable?

@chucker
Copy link

chucker commented May 5, 2022

@wfurt client in my case. macOS as a server is uncommon. But server might still be useful (if less important) for developers who run Kestrel on a Mac?

@wfurt
Copy link
Member Author

wfurt commented May 5, 2022

And yes, the server part would be for Mac developers (like me). Production deployment of Kestrel on macOS is unlikely IMHO.

Just to elaborate more since the use case here is different from #27727.

Aside testing and support, the biggest challenge with using OpenSSL on macOS is integration with keychain and certificate management. Using Apple's AP allows to use private keys that are non-exportable and private keys are not in core dumps. And Apple packages LibreSSL instead of OpenSSL and that is not binary compatible. So dependency management is going to be hard.

The new Apple's api makes it very difficult (and maybe impossible) for SslStream to consume. Based on testing @filipnavara did, client may be easier and there may some possibilities. The challenge would be moving forward with client if there is not solution for the server. That would effectively mean we would need to maintain both PAL layers.

@bhoffman-splunk
Copy link

bhoffman-splunk commented May 6, 2022

@wfurt sorry for not being more clear. This is for clients.

As a client, not being able to make encrypted connections to TLS 1.3-only services/servers is by far the bigger issue. I'm finding companies start a new green-field API or service, and configure it to only use TLS 1.3. Writing .NET code that can talk to services configured like this will only grow with time.

At this point I'm literally shelling out from .NET to run "ncat + openssl" via the Process class to make requests for me and get the response, which is super clumsy and slow.

@wfurt
Copy link
Member Author

wfurt commented May 6, 2022

ok. thanks for the feedback @bhoffman-splunk and @chucker. I'm not sure if this fits to 7.0 but I'll try to take another look and I'll sync up with with @filipnavara.

@filipnavara
Copy link
Member

I am gonna be away for the next week but happy to discuss it when I get back.

therealchjones added a commit to therealchjones/dotnet-tlsprinter that referenced this issue May 19, 2022
(Of note, current .NET 6 won't do TLS 1.3 on macOS. Like, at all. See
dotnet/runtime#1979)

Minor other changes
fixed an error in ExtensionType enum
@davidkaya
Copy link
Contributor

Hi, are there any plans on supporting TLS 1.3 on macOS in future .NET releases?

Our usecase is that we have customers with 10s of thousands of users using our software (.NET 6) on their macOS. The server side is deployed on a Windows Server which has only TLS 1.3 enabled. The software deployed on workstations currently cannot communicate with the server side due to the TLS 1.3 requirement.

@wfurt wfurt modified the milestones: Future, 8.0.0 Nov 16, 2022
@wfurt
Copy link
Member Author

wfurt commented Nov 16, 2022

triage: we should investigate for 8.0

@filipnavara
Copy link
Member

Here's a link to the example on how to "misuse" the framer API in Network.framework to perform client-side TLS: https://gist.github.com/filipnavara/d5fb55bdb5edcceb1981f73078b855c4

(I'll be happy to answer any questions about it once I get back from holiday at the beginning of December.)

@wfurt wfurt modified the milestones: 8.0.0, Future Jun 6, 2023
@karelz karelz modified the milestones: Future, 9.0.0 Jul 18, 2023
@rzikm rzikm modified the milestones: 9.0.0, Future Jun 11, 2024
@pmauchle
Copy link

Does anybody have a workaround for this problem that works in combination with HttpClient and consumes TLS 1.3 APIs?

@wfurt
Copy link
Member Author

wfurt commented Jun 17, 2024

you would need TLS terminating proxy @pmauchle. It is probably doable but tricky. I really wish we can make progress but realistically we don't have resources to get it done in 9.

@stvndall
Copy link

stvndall commented Dec 13, 2024

Azure services are now highlighting that TLS 1.2 needs to be upgraded to TLS 1.3. After a day of not understanding why local development calling to hosted services was suddenly not working, we found the security team upgraded based on Azure's recommendation. Half the team ended up with issues.

This really needs a fix, considering Azure is recommending to upgrade, and our security team says they can select either 1.2 or 1.3, not both. Considering both are Microsoft, this needs a patch before dotnet can run on security recommendations just because team members are running on mac.

@liveans
Copy link
Member

liveans commented Dec 13, 2024

Hello @stvndall,
This is something we're actively working on it right now, and we're targeting to finish this in January.

@karelz karelz modified the milestones: Future, 10.0.0 Dec 17, 2024
@karelz
Copy link
Member

karelz commented Dec 17, 2024

@stvndall can you tell us more about your Mac usage and why are your devs blocked? What are the apps you are using that require only TLS 1.3?
Note that we plan to fix this in upcoming release - .NET 10. The change required to support TLS 1.3 on Mac is a non-trivial feature and we won't be able to backport it into .NET 9.
In the meantime, we would recommend to keep your Azure services on TLS 1.2, which will enable your Mac workloads. The Azure warning is IMHO a bit aggressive on security recommendations and there is not immediate danger with TLS 1.2 being deprecated or compromised security-wise.

Just FYI: The key reason why TLS 1.3 on Mac is not supported yet by .NET, is because Apple decided to not implement it in their low-level APIs (where they support all previous TLS versions). They exposed only new high-level APIs which encapsulate also TCP connection. That model is not working for .NET, because we expose the TCP connection under TLS. Therefore we have to do a lot of hackary around the new Apple APIs, which is complicated.

@stvndall
Copy link

stvndall commented Jan 9, 2025

Hi @karelz.

It is no specific app. We are writing applications that are hosted on azure, with there being many APIs that are in flux, it is not viable for the team to use local environments for the whole system. Because of that, we point to dev environments hosted on Azure from our local development environments.

For security reasons, the infrastructure team set v1.3 as the minimum TLS version, as per Azure constant popup recommendations.

This locked out all members not using Windows (the team is not permitted to use Linux 🤷 ). Any members using the Mac were instantly unable to communicate with services on the dev environments until they were rolled back to v1.2 as minimum. This caused some internal confusion and issues as Azure was constantly reminding our infra team to set the TLS minimum to TLS v1.3

@wfurt
Copy link
Member Author

wfurt commented Jan 13, 2025

I'm wondering where the recommendation is coming from @stvndall? AFAIK the recommendation for Azure is 1.2+ -> and that is what all our services use.

Also from the description it seems like you really need just the client side, right? e.g. functionality from #104835 would be sufficient? It is tricky business and we will be looking for beta testers at some point. Hopefully it will land in 10.0.

@johnnypea
Copy link

This is ridiculous 🙄

@tskardal
Copy link

tskardal commented Mar 4, 2025

I might be missing some pieces here, so sorry in advance if that is. I too am unable to develop against an API that requires TLS 1.3.

I'm wondering where the recommendation is coming from @stvndall? AFAIK the recommendation for Azure is 1.2+ -> and that is what all our services use.

Requiring TLS 1.3 for Azure App Services is at least recommended by Microsoft Defender for Cloud as a Medium risk level.

Also from the description it seems like you really need just the client side, right? e.g. functionality from #104835 would be sufficient? It is tricky business and we will be looking for beta testers at some point. Hopefully it will land in 10.0.

When writing clients for HTTP+JSON APIs you'll end up with this kind of error:

using System.Security.Authentication;

var handler = new HttpClientHandler
{
    // Results in the following exception:
    // Unhandled exception. System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
    // ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
    // ---> System.PlatformNotSupportedException: The requested security protocol is not supported.
    SslProtocols = SslProtocols.Tls13
    
    // This will work (if the server accepts it):
    // SslProtocols = SslProtocols.Tls12
};

using var client = new HttpClient(handler);
var response = await client.GetAsync("https://github.com");
response.EnsureSuccessStatusCode();
var content = await response.Content.ReadAsStringAsync();
Console.WriteLine(content);

@jogibear9988
Copy link

Isn't it possible that netcore uses openssl on other os ?
in https://github.com/dotnetprojects/S7CommPlusDriver/tree/master/src/S7CommPlusDriver/runtimes we use it for ssl, but we have to deliver openssl for every OS in our package. This could be done my the MS netcore installer for each system

@rzikm
Copy link
Member

rzikm commented Mar 17, 2025

@jogibear9988 we want to specifically avoid shipping OpenSSL binaries. Shipping crypto code is always problematic due to various certifications and restrictions. Also delivery of critical security fixes is more difficult for crypto code which is not part of the given platform (e.g. Schannel fixes may be delivered as part of Windows Update, but your deployed application which shipped with openssl library has to be patched manually).

@wfurt
Copy link
Member Author

wfurt commented Mar 17, 2025

One there are challenges for integration with KeyChains. The scenario @tskardal described would be covered by #104835 or some of its reincarnations. The problematic part is server side e.g. dotnet/aspnetcore#60200

Whoever is anxious can either contribute or at least try to convince Apple to provide useable API.

@tverboon
Copy link

Whoever is anxious can either contribute or at least try to convince Apple to provide useable API.

If we Mac developers are going to report bugs through Apple Feedback, could you please provide a design document or a list of requirements that the API needs to meet?

@rzikm
Copy link
Member

rzikm commented Mar 17, 2025

If we Mac developers are going to report bugs through Apple Feedback, could you please provide a design document or a list of requirements that the API needs to meet?

We don't have a formal design document at hand. In short, since SslStream is a wrapper class around another stream, we need to be able to exchange raw bytes with the crypto library implementing the actual TLS logic, i.e. abstract away the network transport. The (very) simplified (client, openssl-like) handshake code is basically

do while not authenticated:
    get outgoing blob to send to server from crypto lib
    receive response blob from the server and pass it to crypto lib
    call a method on crypto lib to progress the handshake, react to any events (received remote cert, remote side asked for cert, ....)

Any api which allows us to perform TLS handshake (and rest of the connection) without involving sockets or network will do. The old Secure Transport API loosely copies OpenSSL API and was very suitable for this. However, the new Network framework is built around the concept of connections between endpoints. So we need to resort to some workarounds like abusing the Framer concept to extract/inject TLS messages from elsewhere.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests