Discussion:
KLEE relationship between loop length and counter.
Kirill Bogdanov
2014-03-25 09:33:26 UTC
Permalink
Hello Everyone,
(Sorry for the previous email, it has been sent by mistake before I
finished it!)

Could you please let me know if what I am doing cannot be achieved with
KLEE and if it can, how can I do it?

1. Please consider my usage example below. The problem that I am observing
is that KLEE does not sees relationship between loop length and the counter
"n".
In my experiments (with different search strategies) KLEE always
encountered assertion by a blind search, and usually after generating 500
test cases.

{
int a;
klee_make_symbolic(&a, sizeof(a), "a");
int i =0 ;
int n =1;
for (i =0; i < a;i++){
n++;
}
if (n == 500)
klee_assert(0);
return 0;
}

Is this is something that is fundamentally impossible in KLEE, or can be
achieved by rewriting my example?


Thank you very much!
Urmas Repinski
2014-03-25 11:12:22 UTC
Permalink
Hi, Kirill.

First of all, it is not clear what type of inputs should be generated.
In the example below all integers will be generated, starting from 0 ... including 500.
klee_assert function will trigger __assert_fail klee function, this possibly will terminate input generation process, possibly not, i am not very sure.

Anyway it is possible that you want something more meaningful, if input 500 should be generated, or input 500 should not be generated.

In this case better to use klee_assume function:

void klee_assume(int constraint) {

if (!constraint) klee_silent_exit(0);

}


{int a;
klee_make_symbolic(&a, sizeof(a), "a");
int i =0 ;
int n =1;
for (i =0; i < a;i++){

n++;
}
klee_assume(n==500); or klee_assume(n!=500);
Will generate a=500 or all a-s, except 500.

Urmas Repinski.

Date: Tue, 25 Mar 2014 09:33:26 +0000
From: kirill.sc-***@public.gmane.org
To: klee-dev-AQ/***@public.gmane.org
Subject: [klee-dev] KLEE relationship between loop length and counter.

Hello Everyone,(Sorry for the previous email, it has been sent by mistake before I finished it!)


Could you please let me know if what I am doing cannot be achieved with KLEE and if it can, how can I do it?

1. Please consider my usage example below. The problem that I am observing is that KLEE does not sees relationship between loop length and the counter "n".
In my experiments (with different search strategies) KLEE always encountered assertion by a blind search, and usually after generating 500 test cases.

{int a;
klee_make_symbolic(&a, sizeof(a), "a");
int i =0 ;
int n =1;
for (i =0; i < a;i++){

n++;
}
if (n == 500)
klee_assert(0);
return 0;
}
Is this is something that is fundamentally impossible in KLEE, or can be achieved by rewriting my example?


Thank you very much!
Urmas Repinski
2014-03-25 13:51:05 UTC
Permalink
Hi Kirill.

The solution is easy actually.

> Is it correct that klee_assume() takes constraint with symbolic variables? While in my example n is not symbolic.
Yes, its correct, use any variables (not symbolic only) in klee_assume, and only inputs, that satisfy assumption, will be generated.

About the second problem i suggest to use klee_assume

void klee_assume(int constraint) {

if (!constraint) klee_silent_exit(0);

}


> What I want to achieve is by applying certain search strategies
(e.g. Depth First) I want KLEE instead of traversing a=1,2,3... but to
give me a=500 as a one of the first test cases. > if I change 500 to 5M I will be waiting forever for the answer while it is right here.

> Let me give you second example to explain what I want to get.
> .....

During the bubble some counter is introduced, cnt1 in your example. As far as i understand this counter is increased every time swap is performed during bubble sort.
All you have to do is simply next to statement

cnt1 = BubbleSort(&a[0] , n);

line

klee_assume(cnt1==30); or with any other value you want to test.

This will make sure, that only inputs, that produce 30 swaps for sorting, will be generated. Others will be skipped.

klee_assume() is actually very useful function, if you want to add to program some function, that does nothing, but exists, use klee_assume(true), if you want to skip some input generation, when certain conditions within the program trigger, use klee_assume(false), not klee_assert as it were suggested.

Hope this is that you need,
Urmas Repinski.

Date: Tue, 25 Mar 2014 13:35:21 +0000
Subject: Re: [klee-dev] KLEE relationship between loop length and counter.
From: kirill.sc-***@public.gmane.org
To: urrimus-***@public.gmane.org
CC: klee-dev-AQ/***@public.gmane.org

Hi Urmas,
Thanks for your answer!
Is it correct that klee_assume() takes constraint with symbolic variables? While in my example n is not symbolic.

What I want to achieve is by applying certain search strategies (e.g. Depth First) I want KLEE instead of traversing a=1,2,3... but to give me a=500 as a one of the first test cases. if I change 500 to 5M I will be waiting forever for the answer while it is right here.


Let me give you second example to explain what I want to get. Imagine that I have a symbolic integer array#define N 10a[N];klee_make_symbolic(&a, sizeof(a), "a");


And I passing it to a bubble sort function, where I counting number of swaps that function performs to sort an array.cnt1 = BubbleSort(&a[0] , n);

Finally, I have a condition, where I comparing number of swaps to some value. Meaning that I want KLEE to come up with an input array such that number of swaps performed by the KLEE is 30 in this example.
if (cnt1 == 30){ klee_assert(0);}
My question is, Can the KLEE see the relationship between symbolic input integer array and the number of swaps that function need to perform.

Thanks!Kirill




On 25 March 2014 11:12, Urmas Repinski <urrimus-***@public.gmane.org> wrote:




Hi, Kirill.

First of all, it is not clear what type of inputs should be generated.
In the example below all integers will be generated, starting from 0 ... including 500.
klee_assert function will trigger __assert_fail klee function, this possibly will terminate input generation process, possibly not, i am not very sure.


Anyway it is possible that you want something more meaningful, if input 500 should be generated, or input 500 should not be generated.

In this case better to use klee_assume function:

void klee_assume(int constraint) {

if (!constraint) klee_silent_exit(0);

}


{int a;
klee_make_symbolic(&a, sizeof(a), "a");
int i =0 ;
int n =1;
for (i =0; i < a;i++){

n++;
}
klee_assume(n==500); or klee_assume(n!=500);
Will generate a=500 or all a-s, except 500.

Urmas Repinski.

Date: Tue, 25 Mar 2014 09:33:26 +0000
From: kirill.sc-***@public.gmane.org

To: klee-dev-AQ/***@public.gmane.org
Subject: [klee-dev] KLEE relationship between loop length and counter.

Hello Everyone,
(Sorry for the previous email, it has been sent by mistake before I finished it!)


Could you please let me know if what I am doing cannot be achieved with KLEE and if it can, how can I do it?


1. Please consider my usage example below. The problem that I am observing is that KLEE does not sees relationship between loop length and the counter "n".

In my experiments (with different search strategies) KLEE always encountered assertion by a blind search, and usually after generating 500 test cases.


{int a;
klee_make_symbolic(&a, sizeof(a), "a");
int i =0 ;
int n =1;
for (i =0; i < a;i++){


n++;
}
if (n == 500)
klee_assert(0);
return 0;
}
Is this is something that is fundamentally impossible in KLEE, or can be achieved by rewriting my example?



Thank you very much!
Kirill Bogdanov
2014-03-25 13:35:21 UTC
Permalink
Hi Urmas,

Thanks for your answer!

Is it correct that klee_assume() takes constraint with symbolic variables?
While in my example n is not symbolic.

What I want to achieve is by applying certain search strategies (e.g. Depth
First) I want KLEE instead of traversing a=1,2,3... but to give me a=500
as a one of the first test cases.
if I change 500 to 5M I will be waiting forever for the answer while it is
right here.


Let me give you second example to explain what I want to get.
Imagine that I have a symbolic integer array
#define N 10
a[N];
klee_make_symbolic(&a, sizeof(a), "a");

And I passing it to a bubble sort function, where I counting number of
swaps that function performs to sort an array.
cnt1 = BubbleSort(&a[0] , n);

Finally, I have a condition, where I comparing number of swaps to some
value. Meaning that I want KLEE to come up with an input array such that
number of swaps performed by the KLEE is 30 in this example.
if (cnt1 == 30){
klee_assert(0);
}

My question is, Can the KLEE see the relationship between symbolic input
integer array and the number of swaps that function need to perform.

Thanks!
Kirill





On 25 March 2014 11:12, Urmas Repinski <urrimus-***@public.gmane.org> wrote:

> Hi, Kirill.
>
> First of all, it is not clear what type of inputs should be generated.
> In the example below all integers will be generated, starting from 0 ...
> including 500.
> klee_assert function will trigger __assert_fail klee function, this
> possibly will terminate input generation process, possibly not, i am not
> very sure.
>
> Anyway it is possible that you want something more meaningful, if input
> 500 should be generated, or input 500 should not be generated.
>
> In this case better to use klee_assume function:
>
> void klee_assume(int constraint) {
> if (!constraint) klee_silent_exit(0);
> }
>
>
> {
> int a;
> klee_make_symbolic(&a, sizeof(a), "a");
> int i =0 ;
> int n =1;
> for (i =0; i < a;i++){
> n++;
> }
> klee_assume(n==500); or klee_assume(n!=500);
>
> Will generate a=500 or all a-s, except 500.
>
> Urmas Repinski.
>
> ------------------------------
> Date: Tue, 25 Mar 2014 09:33:26 +0000
> From: kirill.sc-***@public.gmane.org
> To: klee-dev-AQ/***@public.gmane.org
> Subject: [klee-dev] KLEE relationship between loop length and counter.
>
>
> Hello Everyone,
> (Sorry for the previous email, it has been sent by mistake before I
> finished it!)
>
> Could you please let me know if what I am doing cannot be achieved with
> KLEE and if it can, how can I do it?
>
> 1. Please consider my usage example below. The problem that I am observing
> is that KLEE does not sees relationship between loop length and the counter
> "n".
> In my experiments (with different search strategies) KLEE always
> encountered assertion by a blind search, and usually after generating 500
> test cases.
>
> {
> int a;
> klee_make_symbolic(&a, sizeof(a), "a");
> int i =0 ;
> int n =1;
> for (i =0; i < a;i++){
> n++;
> }
> if (n == 500)
> klee_assert(0);
> return 0;
> }
>
> Is this is something that is fundamentally impossible in KLEE, or can be
> achieved by rewriting my example?
>
>
> Thank you very much!
>
> _______________________________________________ klee-dev mailing list
> klee-dev-AQ/***@public.gmane.org https://mailman.ic.ac.uk/mailman/listinfo/klee-dev
>
Cristian Cadar
2014-03-25 15:32:14 UTC
Permalink
Hi Kirill,

KLEE does not perform any kind of abstraction for loops. In your
example, the order in which paths are explored is dictated by the search
heuristics employed.

Cristian

On 25/03/14 09:33, Kirill Bogdanov wrote:
> Hello Everyone,
> (Sorry for the previous email, it has been sent by mistake before I
> finished it!)
>
> Could you please let me know if what I am doing cannot be achieved with
> KLEE and if it can, how can I do it?
>
> 1. Please consider my usage example below. The problem that I am
> observing is that KLEE does not sees relationship between loop length
> and the counter "n".
> In my experiments (with different search strategies) KLEE always
> encountered assertion by a blind search, and usually after generating
> 500 test cases.
>
> {
> int a;
> klee_make_symbolic(&a, sizeof(a), "a");
> int i =0 ;
> int n =1;
> for (i =0; i < a;i++){
> n++;
> }
> if (n == 500)
> klee_assert(0);
> return 0;
> }
>
> Is this is something that is fundamentally impossible in KLEE, or can be
> achieved by rewriting my example?
>
>
> Thank you very much!
>
>
> _______________________________________________
> klee-dev mailing list
> klee-dev-AQ/***@public.gmane.org
> https://mailman.ic.ac.uk/mailman/listinfo/klee-dev
>
Kirill Bogdanov
2014-03-25 17:02:46 UTC
Permalink
Thank you for your answers,

It is very helpful, but I don't think its entirely answers my question.
Below I have added my entire code of the example (to reduce confusion where
possible)

1. When KLEE comes to the line "if (cnt1 == 30)" what type of the
constraint its generated? Does this constraint mathematically related to
the symbolic array "a" somehow (e.g. are "a" and "cnt" part of the same
expression in STP solver)? Can I print this data constraint?
2. I've been monitoring test cases generated by KLEE under different search
strategies. Generated test cases were not dependant on the value 30 or
"cnt1 < 30" or "cnt1 > 30". KLEE was following the same pattern regardless
of the the condition that I would specify for the assertion. This
observation makes me believe that KLEE "blindly" searches through all
possible input array orderings until it happens to find the one that
satisfies condition for "number of swaps being equal to 30".
3. Different search strategies take different time to find a solution,
which is expected.
4. Time increases exponentially as I increase input array size (#define N
9).

What I am trying to say is that KLEE does not "understand" relationship
between order of the input array and the number of swaps that it takes to
order it .
Is this statement correct?

Practically, is it possible to rearrange my code or add some arguments when
running klee, so that I can find solutions (at least one) in reasonable
time for the arrays of size more than 9 ? (apart from using cloud9 and
parallelizing )


Thank you very much!

#define N 9
int cnt = 0;
int main() {

int i;
int a[N];

klee_make_symbolic(&a, sizeof(a), "a");

int cnt1 = 0;

cnt1 = BubbleSort(&a[0] , N);

if (cnt1 == 30){
printf("Assert\n");
klee_assert(0);
}
printf("%i \n", cnt1);
return 0;
}

//simple bubble sort takes pointer to int array and number of elements
int BubbleSort(int* a, int n){
int swap = 1;
int i =0;
int temp =0;
int cnt = 0;
while (swap){
swap = 0;
for (i= 0; i < n-1; i++){

if (a[i] > a[i+1]){
//swap
temp = a[i];
a[i] = a[i+1];
a[i+1] = temp;
swap = 1;
cnt+=1;
}

}
}

return cnt;
}


On 25 March 2014 15:32, Cristian Cadar <c.cadar-AQ/***@public.gmane.org> wrote:

> Hi Kirill,
>
> KLEE does not perform any kind of abstraction for loops. In your example,
> the order in which paths are explored is dictated by the search heuristics
> employed.
>
> Cristian
>
>
> On 25/03/14 09:33, Kirill Bogdanov wrote:
>
>> Hello Everyone,
>> (Sorry for the previous email, it has been sent by mistake before I
>> finished it!)
>>
>> Could you please let me know if what I am doing cannot be achieved with
>> KLEE and if it can, how can I do it?
>>
>> 1. Please consider my usage example below. The problem that I am
>> observing is that KLEE does not sees relationship between loop length
>> and the counter "n".
>> In my experiments (with different search strategies) KLEE always
>> encountered assertion by a blind search, and usually after generating
>> 500 test cases.
>>
>> {
>> int a;
>> klee_make_symbolic(&a, sizeof(a), "a");
>> int i =0 ;
>> int n =1;
>> for (i =0; i < a;i++){
>> n++;
>> }
>> if (n == 500)
>> klee_assert(0);
>> return 0;
>> }
>>
>> Is this is something that is fundamentally impossible in KLEE, or can be
>> achieved by rewriting my example?
>>
>>
>> Thank you very much!
>>
>>
>> _______________________________________________
>> klee-dev mailing list
>> klee-dev-AQ/***@public.gmane.org
>> https://mailman.ic.ac.uk/mailman/listinfo/klee-dev
>>
>>
Urmas Repinski
2014-03-25 18:14:24 UTC
Permalink
Hi Kirill.

I had already suggested to try to use klee_assume instead of klee_assert.

This should be the solution.

Thank You,
Urmas Repinski.

Date: Tue, 25 Mar 2014 17:02:46 +0000
Subject: Re: [klee-dev] KLEE relationship between loop length and counter.
From: kirill.sc-***@public.gmane.org
To: c.cadar-AQ/***@public.gmane.org; urrimus-***@public.gmane.org
CC: klee-dev-AQ/***@public.gmane.org

Thank you for your answers,
It is very helpful, but I don't think its entirely answers my question. Below I have added my entire code of the example (to reduce confusion where possible)

1. When KLEE comes to the line "if (cnt1 == 30)" what type of the constraint its generated? Does this constraint mathematically related to the symbolic array "a" somehow (e.g. are "a" and "cnt" part of the same expression in STP solver)? Can I print this data constraint?
2. I've been monitoring test cases generated by KLEE under different search strategies. Generated test cases were not dependant on the value 30 or "cnt1 < 30" or "cnt1 > 30". KLEE was following the same pattern regardless of the the condition that I would specify for the assertion. This observation makes me believe that KLEE "blindly" searches through all possible input array orderings until it happens to find the one that satisfies condition for "number of swaps being equal to 30".
3. Different search strategies take different time to find a solution, which is expected. 4. Time increases exponentially as I increase input array size (#define N 9).
What I am trying to say is that KLEE does not "understand" relationship between order of the input array and the number of swaps that it takes to order it .
Is this statement correct?
Practically, is it possible to rearrange my code or add some arguments when running klee, so that I can find solutions (at least one) in reasonable time for the arrays of size more than 9 ? (apart from using cloud9 and parallelizing )

Thank you very much!
#define N 9int cnt = 0; int main() {

int i; int a[N];

klee_make_symbolic(&a, sizeof(a), "a");

int cnt1 = 0;

cnt1 = BubbleSort(&a[0] , N);

if (cnt1 == 30){
printf("Assert\n");

klee_assert(0);
}
printf("%i \n", cnt1);
return 0;
}


//simple bubble sort takes pointer to int array and number of elements
int BubbleSort(int* a, int n){
int swap = 1;
int i =0;

int temp =0;
int cnt = 0;
while (swap){
swap = 0; for (i= 0; i < n-1; i++){


if (a[i] > a[i+1]){ //swap temp = a[i]; a[i] = a[i+1];
a[i+1] = temp; swap = 1; cnt+=1;
}
} }

return cnt;
}



On 25 March 2014 15:32, Cristian Cadar <c.cadar-AQ/***@public.gmane.org> wrote:

Hi Kirill,



KLEE does not perform any kind of abstraction for loops. In your example, the order in which paths are explored is dictated by the search heuristics employed.



Cristian



On 25/03/14 09:33, Kirill Bogdanov wrote:


Hello Everyone,

(Sorry for the previous email, it has been sent by mistake before I

finished it!)



Could you please let me know if what I am doing cannot be achieved with

KLEE and if it can, how can I do it?



1. Please consider my usage example below. The problem that I am

observing is that KLEE does not sees relationship between loop length

and the counter "n".

In my experiments (with different search strategies) KLEE always

encountered assertion by a blind search, and usually after generating

500 test cases.



{

int a;

klee_make_symbolic(&a, sizeof(a), "a");

int i =0 ;

int n =1;

for (i =0; i < a;i++){

n++;

}

if (n == 500)

klee_assert(0);

return 0;

}



Is this is something that is fundamentally impossible in KLEE, or can be

achieved by rewriting my example?





Thank you very much!





_______________________________________________

klee-dev mailing list

klee-dev-AQ/***@public.gmane.org

https://mailman.ic.ac.uk/mailman/listinfo/klee-dev
Loading...