Skip to content
Navigation Menu
Toggle navigation
Sign in
Appearance settings
Platform
AI CODE CREATION
GitHub Copilot
Write better code with AI
GitHub Spark
Build and deploy intelligent apps
GitHub Models
Manage and compare prompts
MCP Registry
New
Integrate external tools
DEVELOPER WORKFLOWS
Actions
Automate any workflow
Codespaces
Instant dev environments
Issues
Plan and track work
Code Review
Manage code changes
APPLICATION SECURITY
GitHub Advanced Security
Find and fix vulnerabilities
Code security
Secure your code as you build
Secret protection
Stop leaks before they start
EXPLORE
Why GitHub
Documentation
Blog
Changelog
Marketplace
View all features
Solutions
BY COMPANY SIZE
Enterprises
Small and medium teams
Startups
Nonprofits
BY USE CASE
App Modernization
DevSecOps
DevOps
CI/CD
View all use cases
BY INDUSTRY
Healthcare
Financial services
Manufacturing
Government
View all industries
View all solutions
Resources
EXPLORE BY TOPIC
AI
Software Development
DevOps
Security
View all topics
EXPLORE BY TYPE
Customer stories
Events & webinars
Ebooks & reports
Business insights
GitHub Skills
SUPPORT & SERVICES
Documentation
Customer support
Community forum
Trust center
Partners
Open Source
COMMUNITY
GitHub Sponsors
Fund open source developers
PROGRAMS
Security Lab
Maintainer Community
Accelerator
Archive Program
REPOSITORIES
Topics
Trending
Collections
Enterprise
ENTERPRISE SOLUTIONS
Enterprise platform
AI-powered developer platform
AVAILABLE ADD-ONS
GitHub Advanced Security
Enterprise-grade security features
Copilot for Business
Enterprise-grade AI features
Premium Support
Enterprise-grade 24/7 support
Pricing
Search or jump to...
Search code, repositories, users, issues, pull requests...
Search syntax tips
Provide feedback
Saved searches
Use saved searches to filter your results more quickly
Sign in
Sign up
Appearance settings
Resetting focus
You signed in with another tab or window.
Reload
to refresh your session.
You signed out in another tab or window.
Reload
to refresh your session.
You switched accounts on another tab or window.
Reload
to refresh your session.
Dismiss alert
{{ message }}
secure-software-engineering
/
TypeEvalPy
Public
Notifications
You must be signed in to change notification settings
Fork
5
Star
38
Code
Issues
1
Pull requests
0
Actions
Projects
0
Security
Uh oh!
There was an error while loading.
Please reload this page
.
Insights
Additional navigation options
Code
Issues
Pull requests
Actions
Projects
Security
Insights
Commits
Branch selector
main
User selector
All users
Datepicker
All time
Commit History
Commits on Nov 13, 2025
Additional benchmark corrections (#13)
Show description for cf218c0
jaltmayerpizzorno
authored
cf218c0
Copy full SHA for cf218c0
Commits on Oct 29, 2025
Merge pull request #12 from plasma-umass/202510-righttyper-support
Show description for a7d57c1
ashwinprasadme
authored
a7d57c1
Copy full SHA for a7d57c1
Merge pull request #11 from plasma-umass/202510-corrections
Show description for 1999930
ashwinprasadme
authored
1999930
Copy full SHA for 1999930
Commits on Oct 16, 2025
- fixed inconsistencies between ground truth descriptions and source code;
jaltmayerpizzorno
committed
8b96c2b
Copy full SHA for 8b96c2b
Commits on Oct 13, 2025
- corrected file name;
jaltmayerpizzorno
committed
4f37d35
Copy full SHA for 4f37d35
- corrected variable type;
jaltmayerpizzorno
committed
1fcb4f9
Copy full SHA for 1fcb4f9
Commits on Oct 12, 2025
Added RightTyper to supported tools table
jaltmayerpizzorno
committed
ee99982
Copy full SHA for ee99982
- first pass at adding RightTyper support;
jaltmayerpizzorno
committed
60705f6
Copy full SHA for 60705f6
Commits on Oct 10, 2025
- fixed ManyTypes4Py dataset URL;
jaltmayerpizzorno
committed
84628b3
Copy full SHA for 84628b3
- corrected types;
jaltmayerpizzorno
committed
32a892a
Copy full SHA for 32a892a
- added call to allow typing by dynamic type inference systems;
jaltmayerpizzorno
committed
92dca4c
Copy full SHA for 92dca4c
- deleting 'bool' from types as adding a bool to an int or a bool
Show description for bd04f85
jaltmayerpizzorno
committed
bd04f85
Copy full SHA for bd04f85
Commits on Oct 9, 2025
- adjusted whitespace so that autogeneration doesn't invalidate col_offset;
jaltmayerpizzorno
committed
92b8b80
Copy full SHA for 92b8b80
- deleted requests for functions that cannot be called from ground truth;
jaltmayerpizzorno
committed
2c6cf0a
Copy full SHA for 2c6cf0a
- corrected types. Note that my_decorator.NewClass is not a valid type, but
Show description for 7befcee
jaltmayerpizzorno
committed
7befcee
Copy full SHA for 7befcee
- added calls to make generated benchmark suite usable with dynamic inference tools;
Show description for bb29a9e
jaltmayerpizzorno
committed
bb29a9e
Copy full SHA for bb29a9e
- fixed inconsistencies between Python sources and ground truth files;
jaltmayerpizzorno
committed
7990076
Copy full SHA for 7990076
Commits on Oct 7, 2025
- added printing out more detail on any docker errors, to facilitate development;
jaltmayerpizzorno
committed
b001ea6
Copy full SHA for b001ea6
- deleted requests for functions that cannot be called from ground truth.
Show description for 98c3268
jaltmayerpizzorno
committed
98c3268
Copy full SHA for 98c3268
- corrected types. Note that my_decorator.NewClass is not a valid type, but
Show description for ac898c2
jaltmayerpizzorno
committed
ac898c2
Copy full SHA for ac898c2
- correctly handle missing files;
jaltmayerpizzorno
committed
6462cce
Copy full SHA for 6462cce
- fixed "missing matches" analysis, which was picking results from
Show description for 2f7c605
jaltmayerpizzorno
committed
2f7c605
Copy full SHA for 2f7c605
Commits on Oct 6, 2025
- moved Point out of function scope, making it actually available as a type;
jaltmayerpizzorno
committed
c6b3c6c
Copy full SHA for c6b3c6c
- added function calls to make function types visible to dynamic type inference tools;
jaltmayerpizzorno
committed
3d333cd
Copy full SHA for 3d333cd
- corrected types;
jaltmayerpizzorno
committed
2d0ab43
Copy full SHA for 2d0ab43
- fixed inconsistencies in ground truth files vs. their corresponding python sources,
Show description for 7dc854d
jaltmayerpizzorno
committed
7dc854d
Copy full SHA for 7dc854d
Commits on May 9, 2025
Merge pull request #10 from secure-software-engineering/feature/update-readme.md
Show description for 7093081
ashwinprasadme
authored
7093081
Copy full SHA for 7093081
Commits on May 6, 2025
fix grammar issue
rashidabhar
committed
eff4287
Copy full SHA for eff4287
Added line breaker
rashidabhar
committed
aae8056
Copy full SHA for aae8056
Updated main readme.md
rashidabhar
committed
dc70e02
Copy full SHA for dc70e02
update main readme.md
rashidabhar
committed
d07bf2f
Copy full SHA for d07bf2f
Commits on May 5, 2025
Merge pull request #7 from secure-software-engineering/bugfix-update-function-name-for-class-function-in-ground-truth
Show description for a6716f2
ashwinprasadme
authored
a6716f2
Copy full SHA for a6716f2
Merge pull request #9 from secure-software-engineering/real-world-benchmark-llms
Show description for be105eb
ashwinprasadme
authored
be105eb
Copy full SHA for be105eb
Commits on Apr 29, 2025
delete unnecessary files
rashidabhar
committed
903e8bf
Copy full SHA for 903e8bf
Merge branch 'main' of https://github.com/secure-software-engineering/TypeEvalPy into real-world-benchmark-llms
rashidabhar
committed
1b22072
Copy full SHA for 1b22072
Pagination
Previous
Next
You can’t perform that action at this time.