Shubham’s Blog

GSoC 2019: Week 12 (The Final Week)

2019-08-21T00:00:00+05:30

The last week of coding period is officially over. A summary of the work done during this week is:

#17379 is now complete and currently under review. I will try to get it merged within this week.
#17392 still needs work. I will try to put a closure to this by the end of week.

#17440 was started. It attempts to add a powerful (but optional) SAT solving engine to SymPy (pycosat). The performance gain for SAT solver is also subtle here: Using this

from sympy import *
from sympy.abc import x
r = random_poly(x, 100, -100, 100)
ans = ask(Q.positive(r), Q.positive(x))

The performance is like

# In master
 |  `- 0.631 check_satisfiability  sympy/assumptions/satask.py:30
 |     `- 0.607 satisfiable  sympy/logic/inference.py:38
 |        `- 0.607 dpll_satisfiable  sympy/logic/algorithms/dpll2.py:21
# With pycosat
 |  `- 0.122 check_satisfiability  sympy/assumptions/satask.py:30
 |     `- 0.098 satisfiable  sympy/logic/inference.py:39
 |        `- 0.096 pycosat_satisfiable  sympy/logic/algorithms/pycosat_wrapper.py:11

It is finished and under review now.

Also, with the end of GSoC 2019, final evaluations have started. I will be writing a final report to the whole project by the end of this week.

So far it has been a great and enriching experience for me. It was my first attempt at GSoC and I am lucky to get such an exposure. I acknowledge that I started with an abstract idea of the project but I now understand both the need and the code of New Assumptions pretty well (thanks to Aaron who wrote the most of it). The system is still in its early phases and needs a lot more work. I am happy to be a part of it and I will be available to work on it.

This is the last weekly report but I will still be contributing to SymPy and open source in general. I will try to write more of such experiences through this portal. Till then, Good bye and thank you!

GSoC 2019: Week 10 and 11

2019-08-13T00:00:00+05:30

So, the second last week of the official coding period is over now. During the last two weeks, I was mostly occupied with on-campus placement drives, hence I couldn’t put up a blog earlier. A summary of my work during these weeks is as follows:

First of all, #17144 is merged 😄. This was a large PR and hence took time to get fully reviewed. With this, the performance of New assumptions comes closer to that of the old system. Currently, queries are evaluated about 20X faster than before.

#17379 attempts to remove SymPy’s costly rcall() from the whole assumptions mechanism. It’s a follow-up from #17144 and the performance gain is subtle for large queries. E.g.

from sympy import *
p = random_poly(x, 50, -50, 50)
print(ask(Q.positive(p), Q.positive(x)))

In the master it takes 4.292 s, out of this 2.483 s is spent in rcall. With this, the time spent is 1.929 s and 0.539 s respectively.

#17392 attempts to make the New Assumptions able to handle queries which involve Relationals. Currently, it works only with simple queries (e.g. ask(x>y, Q.positive(x) & Q.negative(y)) now evaluates True) just like the way old system works. This is a much-awaited functionality for the new system. Also, during this I realized that sathandlers lack many necessary facts. This PR also adds many new facts to the system.

For the last week of coding, my attempt would be to complete both of these PRs and get them merged. Also, I will try to add new facts to sathandlers.

GSoC 2019: Week 9

2019-07-31T00:00:00+05:30

I spent most of this week getting #17144 ready to be merged. I had to change a lot of things from the last attempt. One of such was an attempt on early encoding, I had tried it on Literals. They were eventually going to be encoded so I tried to do this when Literals were created only. But as Aaron suggested, my approach had left encodings in the global space and hence could leak memory. During the week, I tried to attach encoding to the CNF object itself but it would have needed a lot of refactoring, since CNF objects interacted with other such objects. So, after some attempts, at the end I left the encoding to be done at last in EncodedCNF object. Currently, this is ready to be merged.

For the coming weeks, I would try to improve over this.

This was also the week for second monthly evaluation and I feel happy to announce that I passed it. From this week my college has also started but I am still able to give the required time to this project and complete it.

Will keep you updated. Thank you !

GSoC 2019: Week 8

2019-07-23T00:00:00+05:30

Now, the eighth week of official coding period has ended. I would give you a summary of my work during this week.

I spent most of this week polishing and finishing my work on #17144. The performance gain from this PR was very subtle but I had tested this only on the /assumptions/tests of SymPy. During this week’s meeting, Aaron suggested an example that had become slower than before:

p = random_poly(x, 10, -10, 10)
ask(Q.positive(p), Q.positive(x))

This made me realize that my strategy had a flaw. I had converted the smallest expressions into CNF objects and applied Boolean functions (or, and, and not) on them keeping the CNF structure intact. The performance regression was coming from operations or and not. In case of CNF structures, the number of expressions increased exponentially for these operations. Since the test suite didn’t have such examples, all the tests passed without any visible performance issue.

With some research, I found that the best approach to handle this (without any kind of pruning) was to convert these expression into NNF form first. So, I implemented a function to_NNF which works with low-level constructs (SymPy’s to_nnf would have been too costly) and converts an expression into NNF. This is then converted into CNF with ease. It keeps the number of clauses in check and hence also increases the performance outcome. The results are such: the above example now takes about 0.4 s on this PR while it takes about 6 s in master.

Currently, the work on this PR is almost complete and it is under review.

For the next week,

My first priority is to get #17144 merged.
Then, work on optimizing rcall. This is an extension for the above work.

Will keep you updated. Thank you !

GSoC 2019: Week 6 and 7

2019-07-15T00:00:00+05:30

With this ends the seventh week of the official coding period. During the end of 6th week and the beginning of 7th week, I was mostly travelling, so I was not able to write a blog for the sixth week. Instead, I will try to summarize my work during the last two weeks here.

For the last few weeks, I have been focused on optimizing the code of new assumptions to enhance its performance. Most of my work has been exploratory, as Aaron says 😅. Indeed I have dry-run, backtracked, profiled, and ran the same code with a debugger too many times to understand the slow parts and the improvements I can make here and there. Mostly the code is optimized given the class structure of SymPy. But it is also the class structure that is adding up to the performance issues. Already noted in my last blog, classes like And and Or sorts their args, hence take a great amount of time. But other SymPy class constructors also take significant time.

With the success of fifth week’s attempt, I have been desperate to bring down the execution time 😅. Some of the attempts I have made, which are included in #17144, are:

I have modified CNF class, which essentially is a low-level implementation for the cnf of any boolean expression. CNF object holds a set of clauses. These clauses are themselves frozenset of Literal objects. Literal class is being implemented just to reduce the unnecessary creation of Not objects (It takes significant execution time and is called many times).
I have also modified the code of sympify(), it appeared to take more time than expected when the argument is a SymPy object already. Consider this, almost one-third execution time.

In [2]: x = Symbol('x')

In [3]: timeit sympify(x) # before change
601 ns ± 14.6 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [3]: timeit sympify(x) # after change
239 ns ± 11.8 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

Finally, I rewrote to_cnf() for CNF objects. By using mostly Python’s built-ins and removing any SymPy object construction during its execution. The performance gain is quite subtle 😎.

In [1]: from sympy import *

In [2]: from sympy.abc import a,b,c,x,y,z

In [3]: k = (x & y & z) | ( a & b & c)

In [4]: from sympy.logic.boolalg import CNF

# Before
In [5]: timeit CNF.from_prop(k) # It is using to_cnf()
1.41 ms ± 18 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

# after
In [5]: timeit CNF.from_prop(k) # It is using the new to_CNF()
31.5 µs ± 1.48 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

There is definitely a limit to performance we can get with Python. But implementing most of the things in Python built-ins we can definitely make things much faster.

For the upcoming week, I will try to complete the following:

The CNF objects are still not simplified. I have to implement simplification to reduce the number of clauses. These have to be fed into satisfiable at the end, which can benefit from lesser number of clauses.
rcall used with sathandlers is another major portion having high execution time. I will try to work it out.
Clean up the code and make it ready for reviewing.

Apart from that, I will also try to shift my focus towards the other part of my project and write some theory solvers. After all, I need to enhance the assumptions not just make it faster 😎.

Peace

GSoC 2019: Week 5

2019-07-03T00:00:00+05:30

With this the fifth week of the official coding period has ended. I will try to give a brief summary of my work during this week.

I spent most of this week learning the inner working of satask module. It took a lot of debugging to understand the on-going processes efficiently 😅 . My major job was to reduce the unwanted slowing portions in the code. I had #11789 for reference. Some of such performance reducing portions of code were:

A major segment of code was creating And objects unnecessarily. As suggested in #17087 the sorting in the And constructor takes up a significant amount of time. These have to be reduced.
Using SymPy objects is itself a bottleneck for performance. Having a system built over SymPy objects slows things down. Python’s built-in types should be used as much as possible.
A specific segment (used many times in the code) calls rcall over propositions. The rcall which is a recursive process also takes a significant of time.
Also, I have tried to pre-compile results as much as possible.

With the above and some other changes, the overall performance of satask has improved much. I have made a PR over this #17144. For an instance,

Tests	After this PR	In master
test_satask	2.39 s	36.26 s
assumptions/tests	16.74 s	127.21 s

There is still scope for improvement in performance. In the coming week, I will try to work these out. I will also try to improve the performance of ask module.

Also, the first evaluations are over now and I feel happy to announce that I passed it. During the first phase I learnt a lot. In last few weeks I got to explore profiling and got to understand how small segments can influence performance. Before this I felt that I already know the codebase, but in reality I had much to explore. My mentors always gave me a good starting point and a direction over the course of this phase. With the hope to work much better in the coming phases, I take your leave now 😄 .

GSoC 2019: Week 4

2019-06-24T00:00:00+05:30

With this the fourth week and the first phase of GSoC 2019 is over. Here I will give you a brief summary of my progress this week.

I started this week by setting up my workspace for profiling the code related to new assumptions. I am using pyinstrument for that. The results of profiler suggests that a significant amount of time is spent in the to_cnf() function which converts the logical expression into their CNF counterparts, to be used by the SAT solver. Also, since this system is built over the SymPy core, a large amount of this time is spent in the core itself (See the graph here). A possible solution to this is to use constructs at a level lower than the SymPy objects, hence removing the overheads.

Also, as suggested in the last blog, there are various ideas proposed for improving the new assumptions mechanism. Some of them have been implemented to some extent in some PRs. Before proceeding for any new strategies, I need to look into these ideas first. I have started an issue-tree to gather them.

Over the end of the week, I also pushed my work on First Order Logic module at #17069. This work is based on #7608 and extends it by adding Equality to it. Currently, there are test failures and some points to decide. I will try to get it done within this week.

I spent most of this week exploring the profiling and benchmarking of code, and I learnt a lot during this. For the coming week, I will focus on speeding up the code in to_cnf. As suggested by Aaron, this seems a good point to start with. Also, I will be working on the FOL module.

GSoC 2019: Week 3

2019-06-17T00:00:00+05:30

The third week of coding period has ended and as I already stated in the last blog, I couldn’t be much productive this week. Nevertheless, I tried to wrap-up some unfinished works. PR #16956 is ready to merge, just waiting for some last reviews. This PR needs to be supplemented with some fixes in the code of Add class to handle all the issues over Relationals. PR #16978 had one major failing test from the stats module. I tried to debug it ,but due to my unfamiliarity with that part of code I took help from Gagandeep and as his comment suggests, the result this PR is giving is correct and more intuitive. At least, one less issue to handle 😌.

Over the end of the week, I shifted my focus towards New Assumptions. One of my major tasks for this summer is to speed up the satask module. As suggested by Aaron, I searched the repository for issues and PRs over this, and I found a number of such unfinished work. I would need to study these ideas and profile the results before working ahead. I would create a issue-tree over it connecting all such PRs and there respective benchmarks.

For this week, I would focus on the above mentioned task only. Since the first phase is about to end, I would have to speed things up.

GSoC 2019: Week 2

2019-06-13T00:00:00+05:30

The second week of official coding period has concluded. I will give you a brief summary of the week’s work.

I started this week with working on PR #16956, the PR is now passing tests. I will try to get it merged soon. Later this week, I tried to include complex -> finite in the assumptions system. I started PR #16978 over this. This PR changes the assumptions system to work over the complex -> finite clause. All the instances of use of is_complex call has to be modified. As suggested by Aaron, the is_complex call was changed with a disjunction of is_complex and is_infinite calls. Luckily for me, the usage of complex assumption was not much in the codebase 😌. Though it meant less test failures to handle, the core assumptions system (in the old system) is quite complex to understand and debug, and hence handling all the failing tests proved quite difficult. Also, I tried to work on it without including any new assumptions to the system. One important result of this is, zoo.is_complex returns False now.

Apart from that I have worked on to sync these changes in the new assumptions system as well and I will be sending PR over it soon.

For the coming week,

I will start to work on SAT solver and to speed it up. As suggested by Aaron, I will first look into the other unfinished PRs over this.
I will sync the new assumptions with the recent developments in the old system.

With enough said, these PRs made me realize how much the codebase can break with small changes in the assumptions. Also, I know this post is a little bit late(because of bad health). I have some pending work over from the last week and I will try to catch speed as soon as possible.

GSoC 2019: Week 1

2019-06-04T00:00:00+05:30

The first week of official Coding Period is over and it went well 😄.

As proposed, I created a branch over #7608 and went through it locally. I have made certain improvements and I will create appropriate PR for them this week. I intend to get that merged in this week only, if everything goes well. Also, as suggested by Aaron, I looked for profiling tools for optimizing slow parts in the codebase. I will be using pyinstrument and the profiler of PyCharm for this purpose.

Later in this week, I started the PR #16956 which fixes issues #16915 and #16582. Both these issues are related to assumptions handling in Relationals. It broke more code than expected 😅. This shows how embedded the old assumptions are in the codebase. I will try to fix them ASAP.

With the first week wrapped up, in the next week :

I will create an experimental PR to include complex -> finite in the old assumptions. This is going to break zoo.is_complex overall. I will try to do it without including new facts. #16666 shows how difficult such proceedings can be.
I will also create a PR to include complex -> finite in the new assumptions. This should be easier than the former. The new assumptions is still not used much in the codebase 😌.

I will try to shift my focus more towards the new assumptions. But I will also look for any major inconsistency between the assumptions systems.

With this, sign out for the next week.

Community Bonding Period

2019-05-29T00:00:00+05:30

Community bonding period has concluded. Due to my exams and travelling, I was not able to devote much of my time over this. But I managed to review PR #16666 and helped it get merged. With this PR, SymPy comes one step closer to unifying both the Assumption systems. Also, with this I realized how difficult reviewing can be 😅.

I had my first meeting with my mentors, Aaron and Kalevi, on Monday where we decided about the timings of weekly meetings. It was also decided to use gitter as a medium for these meetings. Aaron also gave me some directions about the overall project.

Since the official coding period has begun, I will try to stick to my proposal. These are my plans for the coming week:

As per my proposal, I have to include First Order Logic into SymPy. There is already a PR on this ( #7608). During this week I will work on to improve over it.
Also as suggested by Aaron, I will look into some known performance issues with assumptions.

Along with this, I will also be looking for inconsistencies between the two Assumptions systems.

Sign out for the next week 😄

Beginning GSoC 2019: with SymPy

2019-05-12T00:00:00+05:30

Finally, the result of GSoC-2019 came on 6^th of May, 2019. I am very happy to inform that my proposal on improving the assumptions mechanism has been accepted by SymPy for this year’s GSoC. For those who don’t know, SymPy is an open-sourced Computer Algebra System(CAS) written in Python. It was started by Ondrej Cetrik(@cetrik) in 2006, as a project. Since 2007, SymPy has participated in Google Summer of Code every year. GSoC has given much boost to the development of SymPy’s codebase.

My official mentors will be Aaron Meurer(@asmeurer) and Kelevi Suominen(@jksuom). Aaron(who joined from GSoC) is also the lead developer of SymPy. For the next three months, I will be working on my project under their guidance.

GSoC happens in phases. The official coding period will begin from 27^th of May, 2019. Before that, the first phase is the Community Bonding Period. My plans for this period include:

Setting up a blog and synchronizing it with planet.sympy.org.
Discussing with my mentors about the possible means of communication and weekly meetings.
Planning about the implementation of the project.

I have been contributing to SymPy for the past four months. I am very happy to be a part of it.

Looking forward to a great summer with SymPy.

Disclaimer:There will be a lot of posts after this 😄