sparklyr 1.6: weighted quantile summaries, power iteration clustering, spark_write_rds(), and more

The sparklyr 1.6 release introduces weighted quantile summaries, an R interface to power iteration clustering, spark_write_rds(), as well as a number of dplyr-related improvements.

Related Articles

Python 3.12.0 release candidate 1 released

  I’m pleased to announce the release of Python 3.12 release candidate 1.https://www.python.org/downloads/release/python-3120rc1/This is the first release candidate of Python 3.12.0
This release, 3.12.0rc1, is the penultimate release
preview. Entering the release candidate phase, only reviewed code
changes which are clear bug fixes are allowed between this release
candidate and the final release. The second candidate (and the last
planned release preview) is scheduled for Monday, 2023-09-04, while the
official release of 3.12.0 is scheduled for Monday, 2023-10-02.
There will be no ABI changes from this point forward in the 3.12 series, and the goal is that there will be as few code changes as possible.
Call to action
We strongly encourage maintainers of third-party Python projects to
prepare their projects for 3.12 compatibilities during this phase, and
where necessary publish Python 3.12 wheels on PyPI to be ready for the
final release of 3.12.0. Any binary wheels built against Python
3.12.0rc1 will work with future versions of Python 3.12. As always,
report any issues to the Python bug tracker.
Please keep in mind that this is a preview release and while it’s as close to the final release as we can get it, its use is not recommended for production environments.
Core developers: time to work on documentation now
Are all your changes properly documented?Are they mentioned in What’s New?Did you notice other changes you know of to have insufficient documentation?
Major new features of the 3.12 series, compared to 3.11
New features
More flexible f-string parsing, allowing many things previously disallowed (PEP 701).Support for the buffer protocol in Python code (PEP 688).A new debugging/profiling API (PEP 669).Support for isolated subinterpreters with separate Global Interpreter Locks (PEP 684).Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user.Support for the Linux perf profiler to report Python function names in traces.Many large and small performance improvements (like PEP 709), delivering an estimated 5% overall performance improvementcitation needed.
Type annotations
New type annotation syntax for generic classes (PEP 695).New override decorator for methods (PEP 698).
Deprecations
The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623.In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2).The deprecated smtpd and distutils modules have been removed (see PEP 594 and PEP 632. The setuptools package continues to provide the distutils module.A number of other old, broken and deprecated functions, classes and methods have been removed.Invalid backslash escape sequences in strings now warn with SyntaxWarning instead of DeprecationWarning, making them more visible. (They will become syntax errors in the future.)The internal representation of integers has changed in preparation
for performance enhancements. (This should not affect most users as it
is an internal detail, but it may cause problems for Cython-generated
code.)
(Hey, fellow core developer, if a feature you find important is missing from this list, let Thomas know.)
For more details on the changes to Python 3.12, see What’s new in Python 3.12. The next pre-release of Python 3.12 will be 3.12.0rc2, the final release candidate, currently scheduled for 2023-09-04.
More resources
Online Documentation.PEP 693, the Python 3.12 Release Schedule.Report bugs via GitHub Issues.Help fund Python and its community.Enjoy the new releaseThanks
to all of the many volunteers who help make Python Development and
these releases possible! Please consider supporting our efforts by
volunteering yourself or through organization contributions to the Python Software Foundation.Your release team,Thomas WoutersNed DeilySteve DowerŁukasz Langa

Python 3.12.0 release candidate 3 now available

 I’m pleased to announce the release of Python 3.12 release candidate 3.https://www.python.org/downloads/release/python-3120rc3/ This is the second release candidate of Python 3.12.0This release, 3.12.0rc3, is the absolutely last release preview for Python 3.12.
There will be no ABI changes from this point forward
in the 3.12 series. The intent is for the final release of 3.12.0,
scheduled for Monday, 2023-10-02, to be identical to this release
candidate. This really is the last chance to find critical problems in Python 3.12.Call to action
We strongly encourage maintainers of third-party Python projects to
prepare their projects for 3.12 compatibilities during this phase, and
where necessary publish Python 3.12 wheels on PyPI to be ready for the
final release of 3.12.0. Any binary wheels built against Python
3.12.0rc3 will work with future versions of Python 3.12. As always,
report any issues to the Python bug tracker.
Please keep in mind that this is a preview release and while it’s as close to the final release as we can get it, its use is not recommended for production environments.
Core developers: time to work on documentation now
Are all your changes properly documented?Are they mentioned in What’s New?Did you notice other changes you know of to have insufficient documentation?
 Major new features of the 3.12 series, compared to 3.11
 New features
More flexible f-string parsing, allowing many things previously disallowed (PEP 701).Support for the buffer protocol in Python code (PEP 688).A new debugging/profiling API (PEP 669).Support for isolated subinterpreters with separate Global Interpreter Locks (PEP 684).Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user.Support for the Linux perf profiler to report Python function names in traces.Many large and small performance improvements (like PEP 709 and support for the BOLT binary optimizer), delivering an estimated 5% overall performance improvementcitation needed.
Type annotations
New type annotation syntax for generic classes (PEP 695).New override decorator for methods (PEP 698).
Deprecations
The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623.In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2).The deprecated smtpd and distutils modules have been removed (see PEP 594 and PEP 632. The setuptools package continues to provide the distutils module.A number of other old, broken and deprecated functions, classes and methods have been removed.Invalid backslash escape sequences in strings now warn with SyntaxWarning instead of DeprecationWarning, making them more visible. (They will become syntax errors in the future.)The internal representation of integers has changed in preparation
for performance enhancements. (This should not affect most users as it
is an internal detail, but it may cause problems for Cython-generated
code.)
(Hey, fellow core developer, if a feature you find important is missing from this list, let Thomas know.)
For more details on the changes to Python 3.12, see What’s new in Python 3.12. The next scheduled release of Python 3.12 will be 3.12.0, the final release, currently scheduled for 2023-10-02.
 More resources
Online Documentation.PEP 693, the Python 3.12 Release Schedule.Report bugs via GitHub Issues.Help fund Python and its community. Enjoy the new releaseThanks
to all of the many volunteers who help make Python Development and
these releases possible! Please consider supporting our efforts by
volunteering yourself or through organization contributions to the Python Software Foundation.Your release team,Thomas WoutersNed DeilySteve DowerŁukasz Langa

Python 3.12.0 release candidate 2 now available

I’m pleased to announce the release of Python 3.12 release candidate 2.https://www.python.org/downloads/release/python-3120rc2/ This is the second release candidate of Python 3.12.0This release, 3.12.0rc2, is the last release preview for Python 3.12.
There will be no ABI changes from this point forward
in the 3.12 series. The intent is for the final release of 3.12.0,
scheduled for Monday, 2023-10-02, to be identical to this release
candidate. This is the last chance to find critical problems in Python 3.12.Call to action
We strongly encourage maintainers of third-party Python projects to
prepare their projects for 3.12 compatibilities during this phase, and
where necessary publish Python 3.12 wheels on PyPI to be ready for the
final release of 3.12.0. Any binary wheels built against Python
3.12.0rc2 will work with future versions of Python 3.12. As always,
report any issues to the Python bug tracker.
Please keep in mind that this is a preview release and while it’s as close to the final release as we can get it, its use is not recommended for production environments.
Core developers: time to work on documentation now
Are all your changes properly documented?Are they mentioned in What’s New?Did you notice other changes you know of to have insufficient documentation?
 Major new features of the 3.12 series, compared to 3.11
 New features
More flexible f-string parsing, allowing many things previously disallowed (PEP 701).Support for the buffer protocol in Python code (PEP 688).A new debugging/profiling API (PEP 669).Support for isolated subinterpreters with separate Global Interpreter Locks (PEP 684).Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user.Support for the Linux perf profiler to report Python function names in traces.Many large and small performance improvements (like PEP 709), delivering an estimated 5% overall performance improvementcitation needed.
Type annotations
New type annotation syntax for generic classes (PEP 695).New override decorator for methods (PEP 698).
Deprecations
The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623.In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2).The deprecated smtpd and distutils modules have been removed (see PEP 594 and PEP 632. The setuptools package continues to provide the distutils module.A number of other old, broken and deprecated functions, classes and methods have been removed.Invalid backslash escape sequences in strings now warn with SyntaxWarning instead of DeprecationWarning, making them more visible. (They will become syntax errors in the future.)The internal representation of integers has changed in preparation
for performance enhancements. (This should not affect most users as it
is an internal detail, but it may cause problems for Cython-generated
code.)
(Hey, fellow core developer, if a feature you find important is missing from this list, let Thomas know.)
For more details on the changes to Python 3.12, see What’s new in Python 3.12. The next scheduled release of Python 3.12 will be 3.12.0, the final release, currently scheduled for 2023-10-02.
 More resources
Online Documentation.PEP 693, the Python 3.12 Release Schedule.Report bugs via GitHub Issues.Help fund Python and its community. Enjoy the new releaseThanks
to all of the many volunteers who help make Python Development and
these releases possible! Please consider supporting our efforts by
volunteering yourself or through organization contributions to the Python Software Foundation.Your release team,Thomas WoutersNed DeilySteve DowerŁukasz Langa

Python 3.11.0rc2 is now available

This is the second release candidate of Python 3.11https://www.python.org/downloads/release/python-3110rc2/This release, 3.11.0rc2, is the last preview before the final release of Python 3.11.0 on 2022-10-24.Entering the release candidate phase, only reviewed code changes which are clear bug fixes are allowed between this release candidate and the final release. The second candidate and the last planned release preview is currently planned for Monday, 2022-09-05 while the official release is planned for Monday, 2022-10-24.There will be no ABI changes from this point forward in the 3.11 series and the goal is that there will be as few code changes as possible.Modification of the final releaseDue to the fact that we needed to delay the last release candidate by a week and because of personal scheduling problems I am delaying the final release to 2022-10-24 (three weeks from the original date).Call to actionThe 3.11 branch is now accepting changes for 3.11.1. To maximize stability, the final release will be cut from the v3.11.0rc2 tag. If youneed the release manager (me) to cherry-pick any critical fixes, mark issues as release blockers, and/or add me as a reviewer on a criticalbackport PR on GitHub. To see which changes are currently cherry-picked for inclusion in 3.11.0, look at the short-lived branch-v3.11.0https://github.com/python/cpython/tree/branch-v3.11.0 on GitHub.Core developers: all eyes on the docs now* Are all your changes properly documented?* Did you notice other changes you know of to have insufficient documentation?Community membersWe strongly encourage maintainers of third-party Python projects to prepare their projects for 3.11 compatibilities during this phase. As always, report any issues to the Python bug tracker.Please keep in mind that this is a preview release and its use is **not** recommended for production environments.Major new features of the 3.11 series, compared to 3.10Among the new major new features and changes so far:PEP 657 – Include Fine-Grained Error Locations in TracebacksPEP 654 – Exception Groups and except*PEP 673 – Self TypePEP 646 – Variadic GenericsPEP 680 – tomllib: Support for Parsing TOML in the Standard LibraryPEP 675 – Arbitrary Literal String TypePEP 655 – Marking individual TypedDict items as required or potentially-missingbpo-46752 – Introduce task groups to asyncioPEP 681 – Data Class Transformsbpo-433030– Atomic grouping ((? >…)) and possessive quantifiers (*+, ++, ?+, {m,n}+) are now supported in regular expressions.The Faster Cpython Project is already yielding some exciting results. Python 3.11 is up to 10-60% faster than Python 3.10. On average, we measured a 1.22x speedup on the standard benchmark suite. See Faster CPython for details.(Hey, fellow core developer, if a feature you find important is missing from this list, let Pablo know.)The next release will be the final release of Python 3.11.0, which is currently scheduled for Monday, 2022-10-24.More resourcesOnline DocumentationPEP 664, 3.11 Release ScheduleReport bugs at https://bugs.python.org.Help fund Python and its community.And now for something completely differentIn general relativity, a white hole is a theoretical region of spacetime and singularity that cannot be entered from the outside, although energy-matter, light and information can escape from it. In this sense, it is the reverse of a black hole, which can be entered only from the outside and from which energy-matter, light and information cannot escape. White holes appear in the theory of eternal black holes. In addition to a black hole region in the future, such a solution of the Einstein field equations has a white hole region in its past. This region does not exist for black holes that have formed through gravitational collapse, however, nor are there any observed physical processes through which a white hole could be formed. Supermassive black holes are theoretically predicted to be at the centre of every galaxy and that possibly, a galaxy cannot form without one. Stephen Hawking and others have proposed that these supermassive black holes spawn a supermassive white hole.We hope you enjoy the new releases!Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation.https://www.python.org/psf/Your friendly release team,Ned Deily @nad Steve Dower @steve.dower Pablo Galindo Salgado @pablogsal