Merge pull request #35 from wlmcewen/API-3785

API-3785 date and time function support on fields and datetime values

Author: wlmcewen

CHANGELOG.md

@@ -1,3 +1,11 @@
+v1.0.0, 2016-02-11 ([changes](https://github.com/sparkapi/sparkql/compare/v0.3.20...v1.0.0))
+-------------------
+  * [IMPROVEMENT] function support for fields (delayed resolution). Backing systems must
+    implement necessary function behaviour.
+  * Drop support for ruby 1.8.7. Georuby dropped support several years back and
+    this drop allows us to pick up newer update allows us to stay in sync with
+    that gems development
+
 v0.3.24, 2016-01-05 ([changes](https://github.com/sparkapi/sparkql/compare/v0.3.23...v0.3.24))
 -------------------
 

GRAMMAR.md

@@ -94,6 +94,7 @@ filter.
      field
      	: STANDARD_FIELD
      	| CUSTOM_FIELD
+     	| function
      	;
 ```
 
@@ -140,6 +141,7 @@ Functions may optionally have a comma delimited list of parameters.
      function_arg
        : literal
        | literals
+       | field 
        ;
 ```
 
@@ -187,6 +189,7 @@ Literals only support a single value in a condition
      literal
        : DATE
        | DATETIME
+       | TIME
        | BOOLEAN
        | NULL
        ;
@@ -202,6 +205,7 @@ Functions, and literals that can be used in a range
        | DECIMAL
        | DATE
        | DATETIME
+       | TIME
        | function
        ;
 ```

Gemfile

@@ -1,4 +1,3 @@
-source "http://gems.flexmls.com/" 
 source "http://rubygems.org"
 
 # Specify your gem's dependencies in sparkapi_parser.gemspec

README.md

@@ -1,6 +1,6 @@
 SparkQL query language parser
 =====================
-This gem contains the syntax parser for processing spark api filter queries into manageable 
+This gem contains the syntax parser for processing spark api filter queries into manageable
 expressions. To get an overview of the language syntax-wise, refer to the following files:
 
 * lib/sparkql/parser.y   # BNF Grammar
@@ -12,14 +12,16 @@ Installation
 Add the gem to your gemfile:
 
 Gemfile
-	gem 'sparkql', '~> 0.0.1' 
+	gem 'sparkql', '~> 0.0.1'
 
 When completed, run 'bundle install'.
 
 
 Usage
 -------------
-See test/unit/parser_test.rb for generic parsing examples. In most cases an extended parser is 
+Ruby 1.9 or greater is required.
+
+See test/unit/parser_test.rb for generic parsing examples. In most cases an extended parser is
 needed to do anything of significance, such as the postgres and db2 search implementations in the
 API.
 
@@ -40,12 +42,12 @@ The return value will be an array with one expression element containing the que
 
 Development
 -------------
-The parser is based on racc, a yacc like LR parser that is a part of the ruby runtime.  The grammar 
-is located at lib/sparkql/parser.y and is compiled as part of the test process.  Refer to the 
-Rakefile for details. When modifying the grammar, please checkin BOTH the parser.y and parser.rb 
+The parser is based on racc, a yacc like LR parser that is a part of the ruby runtime.  The grammar
+is located at lib/sparkql/parser.y and is compiled as part of the test process.  Refer to the
+Rakefile for details. When modifying the grammar, please checkin BOTH the parser.y and parser.rb
 files.
 
-Debugging grammar issues can be done by hand using the "racc" command. For example, a dump of the 
+Debugging grammar issues can be done by hand using the "racc" command. For example, a dump of the
 parser states (and conflicts) can be generated via
 
 	racc -o lib/sparkql/parser.rb lib/sparkql/parser.y -v  # see lib/sparkql/parser.output

VERSION

@@ -1 +1 @@
-0.3.24
+1.0.0

lib/sparkql/function_resolver.rb

@@ -10,8 +10,8 @@
 # SUPPORTED_FUNCTIONS which will run validation on the function syntax prior to execution.
 class Sparkql::FunctionResolver
   SECONDS_IN_DAY = 60 * 60 * 24
-  STRFTIME_FORMAT = '%Y-%m-%d'
-  
+  STRFTIME_DATE_FORMAT = '%Y-%m-%d'
+  STRFTIME_TIME_FORMAT = '%H:%M:%S.%N'
   VALID_REGEX_FLAGS = ["", "i"]
   SUPPORTED_FUNCTIONS = {
     :polygon => {
@@ -53,6 +53,16 @@ class Sparkql::FunctionResolver
     :now => { 
       :args => [],
       :return_type => :datetime
+    },
+    :date => { 
+      :args => [[:field,:datetime]],
+      :resolve_for_type => true,
+      :return_type => :date
+    },
+    :time => { 
+      :args => [[:field,:datetime]],
+      :resolve_for_type => true,
+      :return_type => :time
     }
   }
 
@@ -89,7 +99,7 @@ def validate()
 
     count = 0
     @args.each do |arg|
-      unless arg[:type] == total_args[count]
+      unless Array(total_args[count]).include?(arg[:type])
         @errors << Sparkql::ParserError.new(:token => @name, 
           :message => "Function call '#{@name}' has an invalid argument at #{arg[:value]}",
           :status => :fatal )
@@ -99,7 +109,7 @@ def validate()
   end
 
   def return_type
-    supported[@name.to_sym][:return_type]
+    support[@name.to_sym][:return_type]
   end
 
   def errors
@@ -126,8 +136,12 @@ def call()
     fill_in_optional_args.each do |default|
       real_vals << default
     end
-
-    v = self.send(name, *real_vals)
+    method = name
+    if support[name][:resolve_for_type]
+      method_type =  @args.first[:type]
+      method = "#{method}_#{method_type}"
+    end
+    v = self.send(method, *real_vals)
 
     unless v.nil?
       v[:function_name] = @name
@@ -172,7 +186,7 @@ def days(num)
     d = Date.today + num
     {
       :type => :date,
-      :value => d.strftime(STRFTIME_FORMAT)
+      :value => d.strftime(STRFTIME_DATE_FORMAT)
     }
   end
 
@@ -183,20 +197,50 @@ def now()
       :value => Time.now.iso8601
     }
   end
+  
+  def date_field(arg)
+    {
+      :type => :function,
+      :value => "date",
+      :args => [arg]
+    }
+  end
+  
+  def time_field(arg)
+    {
+      :type => :function,
+      :value => "time",
+      :args => [arg]
+    }
+  end
+
+  def date_datetime(dt)
+    {
+      :type => :date,
+      :value => dt.strftime(STRFTIME_DATE_FORMAT)
+    }
+  end
+  
+  def time_datetime(dt)
+    {
+      :type => :time,
+      :value => dt.strftime(STRFTIME_TIME_FORMAT)
+    }
+  end
 
   def months num_months
     d = DateTime.now >> num_months
     {
       :type => :date,
-      :value => d.strftime(STRFTIME_FORMAT)
+      :value => d.strftime(STRFTIME_DATE_FORMAT)
     }
   end
 
   def years num_years
     d = DateTime.now >> (num_years * 12)
     {
       :type => :date,
-      :value => d.strftime(STRFTIME_FORMAT)
+      :value => d.strftime(STRFTIME_DATE_FORMAT)
     }
   end
 

lib/sparkql/lexer.rb

@@ -39,6 +39,8 @@ def shift
         literal :DATETIME, @current_token_value
       when @current_token_value = scan(DATE)
         literal :DATE, @current_token_value
+      when @current_token_value = scan(TIME)
+        literal :TIME, @current_token_value
       when @current_token_value = scan(DECIMAL)
         literal :DECIMAL, @current_token_value
       when @current_token_value = scan(INTEGER)