Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[red-knot] Support callable types and typing.Callable #15382

Open
16 of 19 tasks
carljm opened this issue Jan 9, 2025 · 0 comments · Fixed by #16493
Open
16 of 19 tasks

[red-knot] Support callable types and typing.Callable #15382

carljm opened this issue Jan 9, 2025 · 0 comments · Fixed by #16493
Assignees
Labels
red-knot Multi-file analysis & type inference

Comments

@carljm
Copy link
Contributor

carljm commented Jan 9, 2025

Our "callable type" should be able to represent a callable of arbitrary signature; typing.Callable can only spell a limited subset of those types.

This also includes implementing subtyping/assignability/equivalence for this type.


@carljm carljm added the red-knot Multi-file analysis & type inference label Jan 9, 2025
@carljm carljm added this to the Red Knot Q1 2025 milestone Jan 9, 2025
@dhruvmanila dhruvmanila marked this as a duplicate of #14173 Jan 10, 2025
@dhruvmanila dhruvmanila self-assigned this Jan 10, 2025
dhruvmanila added a commit that referenced this issue Mar 8, 2025
## Summary

Part of #15382

This PR implements a general callable type that wraps around a
`Signature` and it uses that new type to represent `typing.Callable`.

It also implements `Display` support for `Callable`. The format is as:
```
([<arg name>][: <arg type>][ = <default type>], ...) -> <return type>
```

The `/` and `*` separators are added at the correct boundary for
positional-only and keyword-only parameters. Now, as `typing.Callable`
only has positional-only parameters, the rendered signature would be:

```py
Callable[[int, str], None]
# (int, str, /) -> None
```

The `/` separator represents that all the arguments are positional-only.

The relationship methods that check assignability, subtype relationship,
etc. are not yet implemented and will be done so as a follow-up.

## Test Plan

Add test cases for display support for `Signature` and various mdtest
for `typing.Callable`.
@dhruvmanila dhruvmanila linked a pull request Mar 8, 2025 that will close this issue
dhruvmanila added a commit that referenced this issue Mar 11, 2025
## Summary

Part of #15382

This PR adds support for inferring the `lambda` expression and return
the `CallableType`.

Currently, this is only limited to inferring the parameters and a todo
type for the return type.

For posterity, I tried using the `file_expression_type` to infer the
return type of lambda but it would always lead to cycle. The main reason
is that in `infer_parameter_definition`, the default expression is being
inferred using `file_expression_type`, which is correct, but it then

Take the following source code as an example:
```py
lambda x=1: x
```

Here's how the code will flow:
* `infer_scope_types` for the global scope
* `infer_lambda_expression`
* `infer_expression` for the default value `1`
* `file_expression_type` for the return type using the body expression.
This is because the body creates it's own scope
* `infer_scope_types` (lambda body scope)
* `infer_name_load` for the symbol `x` whose visible binding is the
lambda parameter `x`
* `infer_parameter_definition` for parameter `x`
* `file_expression_type` for the default value `1`
* `infer_scope_types` for the global scope because of the default
expression

This will then reach to `infer_definition` for the parameter `x` again
which then creates the cycle.

## Test Plan

Add tests around `lambda` expression inference.
dhruvmanila added a commit that referenced this issue Mar 12, 2025
## Summary

Part of #15382 

This PR adds the check for whether a callable type is fully static or
not.

A callable type is fully static if all of the parameter types are fully
static _and_ the return type is fully static _and_ if it does not use
the gradual form (`...`) for its parameters.

## Test Plan

Update `is_fully_static.md` with callable types.

It seems that currently this test is grouped into either fully static or
not, I think it would be useful to split them up in groups like
callable, etc. I intentionally avoided that in this PR but I'll put up a
PR for an appropriate split.

Note: I've an explicit goal of updating the property tests with the new
callable types once all relations are implemented.
dhruvmanila added a commit that referenced this issue Mar 18, 2025
## Summary

Part of #15382

This PR infers the return type `lambda` expression as `Unknown`. In the
future, it would be more useful to infer the expression type considering
the surrounding context (#16696).

## Test Plan

Update existing test cases from `@todo` to the (verified) return type.
dhruvmanila added a commit that referenced this issue Mar 21, 2025
## Summary

Part of #15382

This PR adds support for checking the subtype relationship between the
two callable types.

The main source of reference used for implementation is
https://typing.python.org/en/latest/spec/callables.html#assignability-rules-for-callables.

The implementation is split into two phases:
1. Check all the positional parameters which includes positional-only,
standard (positional or keyword) and variadic kind
2. Collect all the keywords in a `HashMap` to do the keyword parameters
check via name lookup

For (1), there's a helper struct which is similar to `.zip_longest`
(from `itertools`) except that it allows control over one of the
iterator as that's required when processing a variadic parameter. This
is required because positional parameters needs to be checked as per
their position between the two callable types. The struct also keeps
track of the current iteration element because when the loop is exited
(to move on to the phase 2) the current iteration element would be
carried over to the phase 2 check.

This struct is internal to the `is_subtype_of` method as I don't think
it makes sense to expose it outside. It also allows me to use "self" and
"other" suffixed field names as that's only relevant in that context.

## Test Plan

Add extensive tests in markdown.

Converted all of the code snippets from
https://typing.python.org/en/latest/spec/callables.html#assignability-rules-for-callables
to use `knot_extensions.is_subtype_of` and verified the result.
@dhruvmanila dhruvmanila changed the title [red-knot] support callable types and typing.Callable [red-knot] Support callable types and typing.Callable Mar 21, 2025
dhruvmanila added a commit that referenced this issue Mar 22, 2025
## Summary

Part of #15382

This PR adds support for checking the assignability of two general
callable types.

This is built on top of #16804 by including the gradual parameters check
and accepting a function that performs the check between the two types.

## Test Plan

Update `is_assignable_to.md` with callable types section.
dhruvmanila added a commit that referenced this issue Mar 22, 2025
## Summary

Part of #15382, this PR adds support for calling a variable that's
annotated with `typing.Callable`.

## Test Plan

Add test cases in a new `call/annotation.md` file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
red-knot Multi-file analysis & type inference
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants