Wiki Agenda Contact Version française

Schorr-Waite algorithm, proof via recursion

Schorr-Waite's graph marking algorithm, an alternative proof using recursion


Authors: Martin Clochard

Topics: Ghost code / Graph Algorithms / Pointer Programs

Tools: Why3

References: VerifyThis @ ETAPS 2016

See also: Binary Tree Traversal

see also the index (by topic, by tool, by reference, by year)



Schorr-Waite algorithm for general-size records.

Author: Martin Clochard (Université Paris Sud)

Here the proof is carried out by proving an equivalent recursive program. The recursive program can be justified to be equivalent to the looping one: all side-effects/exception throwing are done by running the loop body (which features a minor modification to exit by exception), so the recursive programs amounts to execute an arbitrary number of time the loop body. It is immediately followed by an absurd statement to enforce the equivalence with the loop which runs its body an infinite number of times. Although the added recursive structure can be seen to be computationally irrelevant, it allows to discharge details about Schorr-Waite stack management through a recursive procedure proof. The method basically make explicit the derecursification used to obtain Schorr-Waite algorithm.

See example verifythis_2016_tree_traversal for details about the technique, applied to a similar algorithm for trees.


Component-as-array memory model, with null pointers and arbitrary-sized memory blocks.

module Memory

  use import int.Int
  use import option.Option
  use import map.Map

Memory locations

  type loc

Null pointer

  constant null : loc

Marks used by Schorr-Waite

  type color =
    | White
    | Black (option int)
  type pmem = map loc (map int loc)

Memory blocks have two parts: a marking part containing in particular Schorr-Waite internal management data, and a sequence of pointers to other memory blocks.

  type memory model {

Associate block size to location.

    block_size : map loc int;

Pointers to other memory blocks.

    mutable accessor : pmem;

Marks.

    mutable colors : map loc color;
  }

Global instance for memory

  val memo : memory

null creation

  val null () : loc ensures { result = null }

null test

  val is_null (l:loc) : bool ensures { result <-> l = null }

Get block size associated to a given location

  val get_block_size (l:loc) : int
    requires { l <> null }
    reads { memo }
    ensures { result = memo.block_size[l] /\ result >= 0 }

Access to a mark

  val get_color (l:loc) : color
    requires { l <> null }
    reads { memo }
    ensures { result = memo.colors[l] }

Set a mark. We also impose the restriction that when a block is marked black, the given index must be coherent with the block size. This impose special treatment for 0-sized memory blocks.

  val set_color (l:loc) (c:color) : unit
    requires { l <> null }
    requires { match c with
      | White -> true
      | Black None -> memo.block_size[l] = 0
      | Black (Some ind) -> 0 <= ind < memo.block_size[l]
      end }
    writes { memo.colors }
    ensures { memo.colors = old (memo.colors[l <- c]) }

Getter/Setter for pointer buffer

  val get_acc (l:loc) (k:int) : loc
    requires { l <> null /\ 0 <= k < memo.block_size[l] }
    reads { memo }
    ensures { result = memo.accessor[l][k] }
  val set_acc (l:loc) (k:int) (d:loc) : unit
    requires { l <> null /\ 0 <= k < memo.block_size[l] }
    writes { memo.accessor }
    ensures { memo.accessor =
      old (memo.accessor[l <- memo.accessor[l][k <- d]]) }

Take ghost snapshots of memory.

  val ghost snapshot_acc () : pmem
    reads { memo }
    ensures { result = memo.accessor }
  val ghost snapshot_colors () : map loc color
    reads { memo }
    ensures { result = memo.colors }

end

Define notions about the memory graph

module GraphShape

  use import int.Int
  use import set.Fset
  use import map.Map
  use import Memory

  predicate black (c:color) = c <> White

Edges

  predicate edge (m:memory) (x y:loc) =
    x <> null /\ (exists n. 0 <= n < m.block_size[x] /\ m.accessor[x][n] = y)

Paths

  inductive path memory (x y:loc) =
    | path_nil : forall m x. path m x x
    | path_cons : forall m x y z. edge m x y /\ path m y z -> path m x z

DFS invariant. For technical reason, it must refer to different parts of the memory at different time. The graph structure is given via the initial memory, but the coloring is given via the current one.

  predicate well_colored_on (graph gray:set loc) (m:memory) (cl:map loc color) =
    subset gray graph /\
    (forall x y. mem x graph /\ edge m x y /\ y <> null /\ black cl[x] ->
      mem x gray \/ black cl[y]) /\
    (forall x. mem x gray -> black cl[x])

Unchanged only concerns the graph shape, not the marks

  predicate unchanged (m1 m2:memory) =
    forall x n. x <> null /\ 0 <= n < m1.block_size[x] ->
      m2.accessor[x][n] = m1.accessor[x][n]

end

Proof of Schorr-Waite algorithm

module SchorrWaite

  use import int.Int
  use import option.Option
  use import set.Fset
  use import map.Map
  use import map.Const
  use import ref.Ref
  use import Memory
  use import GraphShape

  let black (l: loc) : bool
    requires { l <> null }
    reads { memo }
    ensures { result <-> black memo.colors[l] }
  = match get_color l with White -> false | _ -> true end

  exception Stop

  let schorr_waite (root: loc) (ghost graph: set loc) : unit

Root belong to graph (note: for simplicity, the graph set may (and likely does) contain the null pointer.

    requires { mem root graph }

Graph is closed by following edges

    requires { forall l n.
      mem l graph /\ l <> null /\ 0 <= n < memo.block_size[l] ->
      mem memo.accessor[l][n] graph }
    writes { memo.accessor, memo.colors }

The graph starts fully unmarked.

    requires { forall x. mem x graph -> not black memo.colors[x] }

The graph structure is left unchanged.

    ensures { unchanged (old memo) memo }

Every non-null location reachable from the root is marked black.

    ensures { forall x. path (old memo) root x /\ x <> null ->
      black memo.colors[x] }

Every other location is left with its previous color.

    ensures { forall x. not path (old memo) root x /\ x <> null ->
      memo.colors[x] = (old memo).colors[x] }
  =
    'I:
    let t = ref root in
    let p = ref (null ()) in

Schorr-Waite loop body.

    let body () : unit

Loop body specification: mindlessly repeat the underlying code.

      requires { !p <> null /\ (!t = null \/ black memo.colors[!t]) ->
        match memo.colors[!p] with
        | Black (Some m) -> 0 <= m < memo.block_size[!p]
        | _ -> false
        end }
      ensures { old (!p <> null \/ (!t <> null /\ not black memo.colors[!t])) }
      ensures { old (!t <> null /\ not black memo.colors[!t] /\
                     memo.block_size[!t] = 0) ->
        memo.colors = old memo.colors[!t <- Black None] /\
        !t = old !t /\ !p = old (!p) /\ memo.accessor = old memo.accessor
      }
      ensures { old (!t <> null /\ not black memo.colors[!t] /\
                     memo.block_size[!t] > 0) ->
        memo.colors = old memo.colors[!t <- Black (Some 0)] /\
        !t = old memo.accessor[!t][0] /\ !p = old (!t) /\
        memo.accessor = old memo.accessor[!t <- memo.accessor[!t][0 <- !p]] }
      ensures { old (!t = null \/ black memo.colors[!t]) ->
        match old (memo.colors[!p]) with
        | Black (Some m) ->
            let n = m + 1 in
            if n = old (memo.block_size[!p])
            then
              !t = old (!p) /\ !p = (old memo.accessor[!p])[m] /\
              memo.colors = old (memo.colors) /\
              memo.accessor =
                (old memo.accessor)[old !p <-
                                    (old memo.accessor[!p])[m <- old !t]]
            else !p = old !p /\ !t = (old memo.accessor[!p])[n] /\
              memo.colors = (old memo.colors)[old !p <- Black (Some n)] /\
              let pi = (old memo.accessor[!p])[m] in
              memo.accessor =
                (old memo.accessor)[old !p <-
                         (old memo.accessor[!p])[n <- pi][m <- old !t]]
        | _ -> false
        end }
      raises { Stop -> old(!p = null /\ (!t = null \/ black memo.colors[!t])) /\
        memo.colors = old memo.colors /\ memo.accessor = old memo.accessor }
    =

Minor modification to the loop: it exits by exception.

      if is_null !p && (is_null !t || black !t) then raise Stop;
      if is_null !t || black !t then begin
        match get_color !p with
        | Black (Some m) ->
            let s = get_block_size !p in
            let n = m + 1 in
            if n = s then begin (* Pop *)
              let q = !t in
              t := !p;
              p := get_acc !p m;
              set_acc !t m q
            end else begin (* Swing *)
              let q = !t in
              t := get_acc !p n;
              set_acc !p n (get_acc !p m);
              set_acc !p m q;
              set_color !p (Black (Some n))
            end
        | _ -> absurd
        end
      end else
        let s = get_block_size !t in
        if s = 0 then (* Mark & continue *) set_color !t (Black None)
        else begin (* Mark & Push *)
          let q = !p in
          p := !t;
          t := get_acc !t 0;
          set_acc !p 0 q;
          set_color !p (Black (Some 0))
        end
    in
    let rec aux (ghost gray:set loc) : unit
      (* DFS invariant *)
      requires { well_colored_on graph gray (at memo 'I) memo.colors }
      requires { mem !t graph }
      (* Non-marked nodes have unchanged structure with
          respect to the initial one. *)
      requires { forall x n.
        x <> null /\ not black memo.colors[x] /\ 0 <= n < memo.block_size[x] ->
        memo.accessor[x][n] = (at memo.accessor 'I)[x][n] }
      (* 'stack frames' are maintained correctly *)
      ensures { !t = old !t /\ !p = old !p }
      (* Pointer buffer is left unchanged *)
      ensures { unchanged (old memo) memo }
      (* Maintain DFS invariant *)
      ensures { well_colored_on graph gray (at memo 'I) memo.colors }
      (* The top node get marked *)
      ensures { black memo.colors[!t] \/ !t = null }
      (* May not mark unreachable node, neither change marked node. *)
      ensures { forall x.
        x <> null -> not path (at memo 'I) !t x \/ black (old memo.colors)[x] ->
        memo.colors[x] = (old memo.colors)[x] }
      (* Never 'exit' the loop during the recursive calls *)
      raises { Stop -> false }
      (* Terminates because there is a limited number of 'grayable' nodes. *)
      variant { cardinal graph - cardinal gray }
    = 'J:
      if is_null !t || black !t then () else begin
        let s = get_block_size !t in
        let ghost g2 = add !t gray in
        assert { path (at memo 'I) !t !t };
        body (); (* Either push or mark & continue. *)
        if s <> 0 then begin
          for i = 0 to s - 2 do (* Over all sub-blocs... *)
            (* DFS invariant. *)
            invariant { well_colored_on graph g2 (at memo 'I) memo.colors }
            (* Current stack frame invariants *)
            invariant { !p = at !t 'J }
            invariant { !t = (at memo.accessor 'I)[!p][i] }
            invariant { memo.colors[!p] = Black (Some i) }
            invariant { forall j. 0 <= j < s /\ j <> i ->
              memo.accessor[!p][j] = (at memo.accessor 'J)[!p][j] }
            invariant { memo.accessor[!p][i] = at !p 'J }
            (* Outside structure is unchanged. *)
            invariant { forall l n.
              l <> null /\ l <> !p /\ 0 <= n < memo.block_size[l] ->
              memo.accessor[l][n] = (at memo.accessor 'J)[l][n] }
            (* All nodes under !p & before i are either null or marked black. *)
            invariant { forall j. 0 <= j < i ->
              let l = memo.accessor[!p][j] in l = null \/ black memo.colors[l] }
            (* Unreachable/pre-marked blocks do not change. *)
            invariant { forall l. l <> null ->
              not path (at memo 'I) !p l \/ black (at memo.colors 'J)[l] ->
                memo.colors[l] = (at memo.colors 'J)[l] }
            'K:
            aux g2; (* Explore sub-bloc. *)
            body (); (* Swing to next sub-bloc. *)
            assert { !t = (at memo.accessor 'K)[!p][i+1]
              = (at memo.accessor 'J)[!p][i+1] }
          done;
          aux g2; (* Explore last sub-bloc. *)
          body (); (* Pop. *)
        end
      end in
    try aux (ghost empty); (* Explore main bloc *)
        body (); (* Exit *)
        absurd; (* Done. *)
    with Stop ->
      (* Need induction to recover path-based specification. *)
      assert { forall x y m. m = at memo 'I /\ x <> null /\ y <> null /\
        mem x graph /\ black memo.colors[x] ->
        ("induction" path m x y) -> black memo.colors[y] }
    end

end

download ZIP archive

Why3 Proof Results for Project "schorr_waite_via_recursion"

Theory "schorr_waite_via_recursion.SchorrWaite": fully verified in 0.01 s

ObligationsAlt-Ergo (1.01)
1. VC for black0.01
2. VC for schorr_waite---
split_goal_wp
  1. exceptional postcondition0.01
2. precondition0.01
3. exceptional postcondition0.01
4. precondition0.01
5. precondition0.01
6. precondition0.01
7. precondition0.01
8. postcondition0.01
9. postcondition0.01
10. postcondition0.01
11. postcondition0.01
12. precondition0.01
13. precondition0.01
14. precondition0.01
15. precondition0.01
16. precondition0.01
17. precondition0.02
18. postcondition0.01
19. postcondition0.01
20. postcondition0.01
21. postcondition0.01
22. unreachable point0.01
23. unreachable point0.01
24. precondition0.01
25. precondition0.01
26. precondition0.01
27. precondition0.01
28. precondition0.01
29. postcondition0.01
30. postcondition0.02
31. postcondition0.01
32. postcondition0.01
33. precondition0.01
34. precondition0.01
35. precondition0.01
36. precondition0.01
37. precondition0.01
38. precondition0.02
39. postcondition0.02
40. postcondition0.01
41. postcondition0.01
42. postcondition0.02
43. unreachable point0.01
44. unreachable point0.00
45. precondition0.01
46. precondition0.01
47. precondition0.01
48. postcondition0.01
49. postcondition0.02
50. postcondition0.01
51. postcondition0.02
52. precondition0.01
53. precondition0.02
54. precondition0.02
55. precondition0.02
56. postcondition0.01
57. postcondition0.01
58. postcondition0.02
59. postcondition0.02
60. precondition0.01
61. precondition0.01
62. precondition0.02
63. precondition0.02
64. postcondition0.01
65. postcondition0.01
66. postcondition0.01
67. postcondition0.02
68. precondition0.02
69. precondition0.02
70. precondition0.01
71. precondition0.02
72. precondition0.02
73. precondition0.01
74. postcondition0.01
75. postcondition0.01
76. postcondition0.01
77. postcondition0.06
78. unreachable point0.01
79. unreachable point0.01
80. precondition0.01
81. precondition0.01
82. precondition0.01
83. precondition0.02
84. precondition0.01
85. postcondition0.01
86. postcondition0.01
87. postcondition0.01
88. postcondition0.03
89. precondition0.02
90. precondition0.02
91. precondition0.02
92. precondition0.01
93. precondition0.02
94. precondition0.02
95. postcondition0.02
96. postcondition0.02
97. postcondition0.02
98. postcondition0.07
99. unreachable point0.01
100. unreachable point0.01
101. precondition0.01
102. precondition0.01
103. precondition0.01
104. postcondition0.01
105. postcondition0.01
106. postcondition0.01
107. postcondition0.01
108. precondition0.01
109. precondition0.01
110. precondition0.02
111. precondition0.01
112. postcondition0.01
113. postcondition0.01
114. postcondition0.02
115. postcondition0.01
116. postcondition0.01
117. postcondition0.01
118. postcondition0.01
119. precondition0.01
120. postcondition0.01
121. postcondition0.01
122. postcondition0.01
123. precondition0.01
124. assertion0.01
125. precondition0.01
126. variant decrease0.03
127. precondition0.10
128. precondition0.02
129. precondition0.02
130. precondition0.05
131. postcondition0.05
132. postcondition0.61
133. postcondition0.57
134. postcondition0.04
135. postcondition0.32
136. VC for schorr_waite0.02
137. loop invariant init0.09
138. loop invariant init0.02
139. loop invariant init0.01
140. loop invariant init0.02
141. loop invariant init0.01
142. loop invariant init0.01
143. loop invariant init0.01
144. loop invariant init0.01
145. loop invariant init0.02
146. variant decrease0.24
147. precondition0.02
148. precondition0.03
149. precondition0.06
150. precondition0.07
151. assertion0.38
152. loop invariant preservation0.41
153. loop invariant preservation0.07
154. loop invariant preservation0.19
155. loop invariant preservation0.12
156. loop invariant preservation0.78
157. loop invariant preservation0.24
158. loop invariant preservation0.42
159. loop invariant preservation2.93
160. loop invariant preservation1.05
161. VC for schorr_waite0.03
162. variant decrease0.22
163. precondition0.01
164. precondition0.02
165. precondition0.06
166. precondition0.06
167. postcondition0.19
168. postcondition0.47
169. postcondition3.75
170. postcondition0.10
171. postcondition1.00
172. VC for schorr_waite0.01
173. postcondition0.02
174. postcondition0.02
175. postcondition0.02
176. postcondition0.02
177. postcondition0.02
178. VC for schorr_waite0.01
179. precondition0.01
180. precondition0.01
181. precondition0.01
182. unreachable point0.02
183. assertion---
induction_pr
  1. assertion---
simplify_trivial_quantification_in_goal
  1. VC for schorr_waite---
simplify_trivial_quantification_in_goal
  1. VC for schorr_waite0.02
2. assertion---
simplify_trivial_quantification_in_goal
  1. VC for schorr_waite---
simplify_trivial_quantification_in_goal
  1. VC for schorr_waite0.01
184. postcondition0.01
185. postcondition0.02
186. postcondition0.01